Using Rx create a polling request for webservice call

Question

Using Rx in C# I am trying to create a polling request to REST API. The problem which i am facing is that, Observable need to send responses in an order. Means If request A went at X Time and request B went at X + dx time and response of B came before A the Observable expression should ignore or cancel request A.

I have written a sample code which tries to depict the scenario. How can i fix it to get only the latest response and cancel or ignore the previous responses.

 class Program
    {
        static int i = 0;

        static void Main(string[] args)
        {
            GenerateObservableSequence();

            Console.ReadLine();
        }

        private static void GenerateObservableSequence()
        {
            var timerData = Observable.Timer(TimeSpan.Zero,
                TimeSpan.FromSeconds(1));

            var asyncCall = Observable.FromAsync<int>(() =>
            {
                TaskCompletionSource<int> t = new TaskCompletionSource<int>();
                i++;

                int k = i;
                var rndNo = new Random().Next(3, 10);
                Task.Delay(TimeSpan.FromSeconds(rndNo)).ContinueWith(r => { t.SetResult(k); });
                return t.Task;
            });

            var obs = from t in timerData
            from data in asyncCall
            select data;

            var hot = obs.Publish();
            hot.Connect();

                hot.Subscribe(j => 
            {
                Console.WriteLine("{0}", j);
            });
        }
    }

After @Enigmativity answer: Added Polling Aync function to always take the latest response:

 public static IObservable<T> PollingAync<T> (Func<Task<T>> AsyncCall, double TimerDuration)
        {
            return Observable
         .Create<T>(o =>
         {
             var z = 0L;
             return
                 Observable
                     .Timer(TimeSpan.Zero, TimeSpan.FromSeconds(TimerDuration))
                     .SelectMany(nr =>
                         Observable.FromAsync<T>(AsyncCall),
                         (nr, obj) => new { nr, obj})
                     .Do(res => z = Math.Max(z, res.nr))
                     .Where(res => res.nr >= z)
                     .Select(res => res.obj)
                     .Subscribe(o);
         });

    }

Lee Campbell · Answer 1 · 2016-02-10T07:00:24.150

This is a common scenario and can be fixed simply.

The key part of your sample code in question is

var obs = from t in timerData
          from data in asyncCall
          select data;

This can be read as "for each value in timerData get all the values in asyncCall". This is the SelectMany (or FlatMap) operator. The SelectMany operator will take all values from the inner sequence (asyncCall) and return their values as they recieved. This means you can get out of order values.

What you want is to cancel a previous inner sequence when the outer sequence (timerData) produces a new value. To do this we want to use the Switch operator instead.

var obs = timerData.Select(_=>asyncCall)
                   .Switch();

Full code could be cleaned to the following. (removed redundant Publish/Connect, dispose of subscription on key press)

class Program { static int i = 0;

    static void Main(string[] args)
    {
        using (GenerateObservableSequence().Subscribe(x => Console.WriteLine(x)))
        {
            Console.ReadLine();
        }
    }

    private static IObservable<int> GenerateObservableSequence()
    {
        var timerData = Observable.Timer(TimeSpan.Zero,
            TimeSpan.FromSeconds(1));

        var asyncCall = Observable.FromAsync<int>(() =>
        {
            TaskCompletionSource<int> t = new TaskCompletionSource<int>();
            i++;

            int k = i;
            var rndNo = new Random().Next(3, 10);
            Task.Delay(TimeSpan.FromSeconds(rndNo)).ContinueWith(r => { t.SetResult(k); });
            return t.Task;
        });

        return from t in timerData
               from data in asyncCall
               select data;
    }
}

--EDIT--

It looks like I have misunderstood the question. And @Enigmativity has provided a more accurate answer. This is a clean up of his answer.

//Probably should be a field?
var rnd = new Random();
var obs = Observable.Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1))
        //.Select(n => new { n, r = ++i })
        //No need for the `i` counter. Rx does this for us with this overload of `Select`
        .Select((val, idx) => new { Value = val, Index = idx})
        .SelectMany(nr =>
            Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10))),
            (nr, _) => nr)
        //.Do(nr => z = Math.Max(z, nr.n))
        //.Where(nr => nr.n >= z)
        //Replace external State and Do with scan and Distinct
        .Scan(new { Value = 0L, Index = -1 }, (prev, cur) => {
            return cur.Index > prev.Index
                ? cur
                : prev;
        })
        .DistinctUntilChanged()
        .Select(nr => nr.Value)
        .Dump();

Isn't this always taking the value from the latest request only? My understanding from the question is that if two calls to `asyncCall` happen and the results come back in call order then the two results are all valid, but if the second call comes back first then ignore the result from the first call. — Enigmativity, Feb 09 '16 at 09:29
Based on your understanding of the question, then your assumption is correct too. — Lee Campbell, Feb 10 '16 at 06:48
@LeeCampbell any suggestion to how to handler exceptions of the previous calls that we have ignored. Instead of Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10)) I will call a async function as shown in the updated question. This async func may throw an exception. Hence I wish to cancel the previous async call and handle exception without ending the IObservable stream. — Balraj Singh, Feb 10 '16 at 09:05
In general, Rx is about composing sequences, and once a sequence has errored, it is considered terminal. So in this sense, your use case is not supported. However, in this scenario you could use a `Catch` operator and yield an appropriate value from there. See http://introtorx.com/Content/v1.0.10621.0/11_AdvancedErrorHandling.html — Lee Campbell, Feb 11 '16 at 01:21

score 2 · Accepted Answer · answered Feb 09 '16 at 09:58

Let's start by simplifying your code.

This is basically the same code:

var rnd = new Random();

var i = 0;

var obs =
    from n in Observable.Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1))
    let r = ++i
    from t in Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10)))
    select r;

obs.Subscribe(Console.WriteLine);

I get this kind of result:

Alternatively, this can be written as:

var obs =
    Observable
        .Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1))
        .Select(n => ++i)
        .SelectMany(n =>
            Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10))), (n, _) => n);

So, now for your requirement:

If request A went at X Time and request B went at X + dx time and response of B came before A the Observable expression should ignore or cancel request A.

Here's the code:

var rnd = new Random();

var i = 0;
var z = 0L;

var obs =
    Observable
        .Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1))
        .Select(n => new { n, r = ++i })
        .SelectMany(nr =>
            Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10))), (nr, _) => nr)
        .Do(nr => z = Math.Max(z, nr.n))
        .Where(nr => nr.n >= z)
        .Select(nr => nr.r);

I don't like using .Do like that, but I can't think of an alternative yet.

This gives this kind of thing:

Notice that the values are only ascending.

Now, you really should use Observable.Create to encapsulate the state that you're using. So your final observable should look like this:

var obs =
    Observable
        .Create<int>(o =>
        {
            var rnd = new Random();
            var i = 0;
            var z = 0L;
            return
                Observable
                    .Timer(TimeSpan.Zero, TimeSpan.FromSeconds(1))
                    .Select(n => new { n, r = ++i })
                    .SelectMany(nr =>
                        Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10))),
                        (nr, _) => nr)
                    .Do(nr => z = Math.Max(z, nr.n))
                    .Where(nr => nr.n >= z)
                    .Select(nr => nr.r)
                    .Subscribe(o);
        });

How would we avoid exceptions coming from older responses which have been filtered using where clause? — Balraj Singh, Feb 09 '16 at 13:31
@BalrajSingh - So, if I were to replace the `++i` with an operation that occasionally threw an error? Is that what is a possible scenario? What kind of error could occur? — Enigmativity, Feb 09 '16 at 20:55
I mean instead of Observable.Timer(TimeSpan.FromSeconds(rnd.Next(3, 10)) I will replace this by an async call to web service and it may throw any type of http error. In that case i need to cancel all older request that i make so that it may not through any error and if it throws then I should have a way to handle it without ending my Observable stream. — Balraj Singh, Feb 10 '16 at 06:08
I have updated the question with sample code for calling any async function and taking the latest response only. This may throw an exception if the async func does. — Balraj Singh, Feb 10 '16 at 06:11
Enigmativity, I have provided an implementation of your answer without the external variables and the `Do` statement. Good job on think out side of the box. — Lee Campbell, Feb 10 '16 at 07:01

Using Rx create a polling request for webservice call

2 Answers2