RX terminolgy: Async processing in RX operator when there are frequent observable notifications

Question

The purpose is to do some async work on a scarce resource in a RX operator, Select for example. Issues arise when observable notifications came at a rate that is faster than the time it takes for the async operation to complete.

Now I actually solved the problem. My question would be what is the correct terminology for this particular kind of issue? Does it have a name? Is it backpressure? Research I did until now indicate that this is some kind of a pressure problem, but not necessarily backpressure from my understanding. The most relevant resources I found are these: https://github.com/ReactiveX/RxJava/wiki/Backpressure-(2.0) http://reactivex.io/documentation/operators/backpressure.html

Now to the actual code. Suppose there is a scarce resource and it's consumer. In this case exception is thrown when resource is in use. Please note that this code should not be changed.

public class ScarceResource
{
    private static bool inUse = false;

    public async Task<int> AccessResource()
    {
        if (inUse) throw new Exception("Resource is alredy in use");

        var result = await Task.Run(() =>
        {
            inUse = true;
            Random random = new Random();
            Thread.Sleep(random.Next(1, 2) * 1000);
            inUse = false;

            return random.Next(1, 10);
        });

        return result;
    }
}

public class ResourceConsumer
{
    public IObservable<int> DoWork()
    {
        var resource = new ScarceResource();
        return resource.AccessResource().ToObservable();
    }
}

Now here is the problem with a naive implementation to consume the resource. Error is thrown because notifications came at a faster rate than the consumer takes to run.

private static void RunIntoIssue()
{
    var numbers = Enumerable.Range(1, 10);
    var observableSequence = numbers
        .ToObservable()
        .SelectMany(n =>
        {
            Console.WriteLine("In observable: {0}", n);
            var resourceConsumer = new ResourceConsumer();
            return resourceConsumer.DoWork();
        });

    observableSequence.Subscribe(n => Console.WriteLine("In observer: {0}", n));
}

With the following code the problem is solved. I slow down processing by using a completed BehaviorSubject in conjunction with the Zip operator. Essentially what this code does is to take a sequential approach instead of a parallel one.

private static void RunWithZip()
{
    var completed = new BehaviorSubject<bool>(true);
    var numbers = Enumerable.Range(1, 10);
    var observableSequence = numbers
        .ToObservable()
        .Zip(completed, (n, c) =>
        {
            Console.WriteLine("In observable: {0}, completed: {1}", n, c);
            var resourceConsumer = new ResourceConsumer();
            return resourceConsumer.DoWork();
        })
        .Switch()
        .Select(n =>
        {
            completed.OnNext(true);
            return n;
        });

    observableSequence.Subscribe(n => Console.WriteLine("In observer: {0}", n));
    Console.Read();
}

Question Is this backpressure, and if not does it have another terminology associated?

score 2 · Accepted Answer · answered Mar 22 '18 at 13:32

You're basically implementing a form of locking, or a mutex. Your code an cause backpressure, it's not really handling it.

Imagine if your source wasn't a generator function, but rather a series of data pushes. The data pushes arrive at a constant rate of every millisecond. It takes you 10 Millis to process each one, and your code forces serial processing. This causes backpressure: Zip will queue up the unprocessed datapushes infinitely until you run out of memory.

RX terminolgy: Async processing in RX operator when there are frequent observable notifications

1 Answers1