WebClient Timeout takes longer than expected (Using: Rx, Try Catch, Task)

Question

Problem: I inherited WebClient in ExtendedWebClient where I override the WebRequest's timeout property in the GetWebRequest method. If I set it to 100ms, or even 20ms, it always takes up to more than 30 seconds at least. Sometimes it seems to not get through at all.

Also, when the service (see code below) serving the images comes back online again, the code written in Rx / System.Reactive does not push images into the pictureBox anymore?

How can I get around this, what am I doing wrong? (See code below)

---

Test case: I have a WinForms test project set up for this, which is doing the following.

1. GetNextImageAsync

   public async Task<Image> GetNextImageAsync()
   {            
       Image image = default(Image);
   
       try {
           using (var webClient = new ExtendedWebClient()) {                    
               var data = await webClient.DownloadDataTaskAsync(new Uri("http://SOMEIPADDRESS/x/y/GetJpegImage.cgi"));
               image = ByteArrayToImage(data);
   
               return image;
           }
       } catch {
           return image;
       }
   }
   

2. ExtendedWebClient

   private class ExtendedWebClient : WebClient
   {
       protected override WebRequest GetWebRequest(Uri address)
       {                
           var webRequest = base.GetWebRequest(address);
           webRequest.Timeout = 100;
           return webRequest;
       }
   }
   

3.1 Presentation code (using Rx)

Note: It has actually never reached the "else" statement in the pictures.Subscribe() body.

    var pictures = Observable
            .FromAsync<Image>(GetNextImageAsync)
            .Throttle(TimeSpan.FromSeconds(.5))
            .Repeat()
            ;

    pictures.Subscribe(img => {
        if (img != null) {
            pictureBox1.Image = img;
        } else {
            if (pictureBox1.Created && this.Created) {
                using (var g = pictureBox1.CreateGraphics()) {
                    g.DrawString("[-]", new Font("Verdana", 8), Brushes.Red, new PointF(8, 8));
                }
            }
        }
    });

3.2 Presentation code (using Task.Run)

Note 1: Here the "else" body is getting called, though WebClient still takes longer than expected to timeout....

Note 2: I don't want to use this method, because this way I can't "Throttle" the image stream, I'm not able to get them in proper order, and do other stuff with my stream of images... But this is just example code of it working...

    Task.Run(() => {
        while (true) {
            GetNextImageAsync().ContinueWith(img => {
                if(img.Result != null) {
                    pictureBox1.Image = img.Result;
                } else {
                    if (pictureBox1.Created && this.Created) {
                        using (var g = pictureBox1.CreateGraphics()) {
                            g.DrawString("[-]", new Font("Verdana", 8), Brushes.Red, new PointF(8, 8));
                        }
                    }
                }
            });                    
        }
    });

4. As reference, the code to tranfser the byte[] to the Image object.

   public Image ByteArrayToImage(byte[] byteArrayIn)
   {
       using(var memoryStream = new MemoryStream(byteArrayIn)){
           Image returnImage = Image.FromStream(memoryStream);
           return returnImage;
       }
   }
   

Answer 1

The Other Problem...

I'll address cancellation below, but there is also a misunderstanding of the behaviour of the following code, which is going to cause issues regardless of the cancellation problem:

var pictures = Observable
  .FromAsync<Image>(GetNextImageAsync)
  .Throttle(TimeSpan.FromSeconds(.5))
  .Repeat()

You probably think that the Throttle here will limit the invocation rate of GetNextImageAsync. Sadly that is not the case. Consider the following code:

var xs = Observable.Return(1)
                   .Throttle(TimeSpan.FromSeconds(5));

How long do you think it will take the OnNext(1) to be invoked on a subscriber? If you thought 5 seconds, you'd be wrong. Since Observable.Return sends an OnCompleted immediately after its OnNext(1) the Throttle deduces that there are no more events that could possibly throttle the OnNext(1) and it emits it immediately.

Contrast with this code where we create a non-terminating stream:

var xs = Observable.Never<int>().StartWith(1)
                   .Throttle(TimeSpan.FromSeconds(5));

Now the OnNext(1) arrives after 5 seconds.

The upshot of all this is that your indefinite Repeat is going to batter your code, requesting images as fast as they arrive - how exactly this is causing the effects you are witnessing would take further analysis.

There are several constructions to limit the rate of querying, depending on your requirements. One is to simply append an empty delay to your result:

var xs = Observable.FromAsync(GetValueAsync)                       
                   .Concat(Observable.Never<int>()
                        .Timeout(TimeSpan.FromSeconds(5),
                                 Observable.Empty<int>()))
                   .Repeat();

Here you would replace int with the type returned by GetValueAsync.

Cancellation

As @StephenCleary observed, setting the Timeout property of WebRequest will only work on synchronous requests. Having looked at this necessary changes to implement cancellation cleanly with WebClient, I have to concur it's such a faff with WebClient you are far better off converting to HttpClient if at all possible.

Sadly, even then the "easy" methods to pull back data such as GetByteArrayAsync don't (for some bizarre reason) have an overload accepting a CancellationToken.

If you do use HttpClient then one option for timeout handling is via the Rx like this:

void Main()
{
    Observable.FromAsync(GetNextImageAsync)
            .Timeout(TimeSpan.FromSeconds(1), Observable.Empty<byte[]>())
            .Subscribe(Console.WriteLine);
}

public async Task<byte[]> GetNextImageAsync(CancellationToken ct)
{
    using(var wc = new HttpClient())
    {
        var response = await wc.GetAsync(new Uri("http://www.google.com"),
            HttpCompletionOption.ResponseContentRead, ct);
        return await response.Content.ReadAsByteArrayAsync();
    }
}

Here I have used the Timeout operator to cause an empty stream to be emitted in the event of timeout - other options are available depending on what you need.

When Timeout does timeout it will cancel it's subscription to FromAsync which in turn will cancel the cancellation token it passes indirectly to HttpClient.GetAsync via GetNextImageAsync.

You could use a similar construction to call Abort on a WebRequest too, but as I said, it's a lot more of a faff given there's no direct support for cancellation tokens.

Answer 2

To quote the MSDN docs:

The Timeout property affects only synchronous requests made with the GetResponse method. To time out asynchronous requests, use the Abort method.

You could mess around with the Abort method, but it's easier to convert from WebClient to HttpClient, which was designed with asynchronous operations in mind.