Buffering of observable to stabilize variable delays for slower observer

Question

I have one observable that produces a sequence of numbers with delays in between the numbers that range from 0 to 1 second (randomly):

var random = new Random();

var randomDelaysObservable = Observable.Create<int>(async observer =>
{
    var value = 0;

    while (true)
    {
        // delay from 0 to 1 second
        var randomDelay = TimeSpan.FromSeconds(random.NextDouble());
        await Task.Delay(randomDelay);

        observer.OnNext(value++);
    }

    return Disposable.Empty;
    // ReSharper disable once FunctionNeverReturns
});

I would like to have a consumer that consumes those numbers and writes them out to the console, but only to take a single number every 2 seconds (exactly every two seconds).

Right now, I have this code for the observer (although I know it isn't correct with the use of await):

var delayedConsoleWritingObserver = Observer.Create<int>(async value =>
{
    // fixed delay of 2 seconds
    var fixedDelay = TimeSpan.FromSeconds(2);
    await Task.Delay(fixedDelay);

    Console.WriteLine($"[{DateTime.Now:O}] Received value: {value}.");
});


randomDelaysObservable.Subscribe(delayedConsoleWritingObserver);

---

If the producer produces numbers every 0 to 1 second, and the consumer is only able to consume single number every 2 seconds, it's clear that the producer produces the numbers faster than the consumer can consume them (backpressure). What I would like to do is to be able to "preload" e.g. 10 or 20 of the numbers from the producer in advance (if the consumer cannot process them fast enough) so that the consumer could consume them without the random delays (but not all of them as the observable sequence is infinite, and we'd run out of memory if it was running for some time).

This would sort of stabilize the variable delays from the producer if I have a slower consumer. However, I cannot think of a possible solution how to do this with the operators in ReactiveX, I've looked at the documentation of Buffer, Sample, Debounce and Window, and none of them look like the thing I'm looking for.

Any ideas on how this would be possible? Please note that even my observer code isn't really correct with the async/await, but I wasn't able to think of a better way to illustrate what I'm trying to achieve.

---

EDIT:

As pointed out by Schlomo, I might not have formulated the question well, as it looks a bit more like an XY-problem. I'm sorry about that. I'll try to illustrate the process I'm trying to model on another example. I don't really care that much about exact time delays on the producer or consumer side. The time delays were really just a placeholder for some asynchronous work that takes some time.

I'm more thinking about a general pattern, where producer produces items at some variable rate, I want to process all the items, and consumer also can only consume the items at some variable rate. And I'm trying to do this more effectively.

I'll try to illustrate this on a more real-world example of a pizza place .

• Let's say that I'm the owner of a pizza place, and we serve just one kind of pizza, e.g. pizza Margherita.
• I have one cook employed in the kitchen who makes the pizzas.
• Whenever an order comes in for a pizza, he takes the order and prepares the pizza.

Now when I look at this as the owner, I see that it's not efficient. Every time a new order comes in, he has to start preparing the pizza. I think we can increase the throughput and serve the orders faster.

We only make one kind of pizza. I'm thinking that maybe if the cook has free time on his hands and there are no currently pending orders, he could prepare a couple of pizzas in advance. Let's say I'd let him prepare up to 10 pizzas in advance -- again, only when he has free time and is not busy fulfilling orders.

When we open the place in the morning, we've got no pizzas prepared in advance, and we just serve the orders as they come in. As soon as there's just a little bit of time and no orders are pending, the cook starts putting the pizzas aside in a queue. And he only stops once there are 10 pizzas in the queue. If there is an incoming order, we just fulfill it from the queue, and the cook needs to fill in the queue from the other end. For example, if we've got the queue completely filled with all 10 pizzas, and we take 1 pizza out, leaving 9 pizzas in the queue, the cook should immediately start preparing the 1 pizza to fill the queue again to 10 pizzas.

I see the generalized problem as a producer-consumer where the producer produces each item in some time, and consumer consumes each item in some time. And by adding this "buffer queue" between them, we can improve the throughput, so they wouldn't have to wait for each other that much. But I want to limit the size of the queue to 10 to avoid making too many pizzas in advance.

Now to the possible operators from Rx:

• Throttle and Sample won't work because they are discarding items produced by the producer. Throughout the day, I don't want to throw away any pizzas that the cook makes. Maybe at the end of the day if few uneaten pizzas are left, it's ok, but I don't want to discard anything during the day.
• Buffer won't work because that would just basically mean to prepare the pizzas in batches of 10. That's not what I want to do because I would still need to wait for every batch of 10 pizzas whenever the previous batch is gone. Also, I would still need to prepare the first batch of 10 pizzas first thing in the morning, and I couldn't just start fulfilling orders. So if there would be 10 people waiting in line before the place opens, I would serve all those 10 people at once. That's not how I want it to work, I want "first come first served" as soon as possible.
• Window is a little bit better than Buffer in this sense, but I still don't think it works completely like the queue that I described above. Again, when the queue is filled with 10 pizzas, and one pizza gets out, I immediately want to start producing new pizza to fill the queue again, not wait until all 10 pizzas are out.

Hope this helps in illustrating my idea a little bit better. If it's still not clear, maybe I can come up with some better code samples and start a new question later.

Answer 1

Here's what your observables could look like using pure Rx:

var producer = Observable.Generate(
    (r: new Random(), i: 0),                    // initial state
    _ => true,                                  // condition
    t => (t.r, t.i + 1),                        // iterator
    t => t.i,                                   // result selector
    t => TimeSpan.FromSeconds(t.r.NextDouble()) // timespan generator
);

var consumer = producer.Zip(
    Observable.Interval(TimeSpan.FromSeconds(2)),
    (_, i) => i
);

However, that isn't an easy thing to 'grab the first n without delay'. So we can instead create a non-time-gapped producer:

var rawProducer = Observable.Range(0, int.MaxValue);

then create the time gaps separately:

var timeGaps = Observable.Repeat(TimeSpan.Zero).Take(10) //or 20
    .Concat(Observable.Generate(new Random(), r => true, r => r, r => TimeSpan.FromSeconds(r.NextDouble())));

then combine those two:

var timeGappedProducer = rawProducer.Zip(timeGaps, (i, ts) => Observable.Return(i).Delay(ts))
    .Concat();

the consumer looks basically the same:

var lessPressureConsumer = timeGappedProducer .Zip(
    Observable.Interval(TimeSpan.FromSeconds(2)),
    (_, i) => i
);

---

Given all of that, I don't really understand why you want to do this. It's not a good way to handle back-pressure, and the question sounds like a bit of an XY-problem. The operators you mention (Sample, Throttle, etc.) are better ways of handling back-pressure.

Answer 2

Your problem as described is well suited to a simple bounded buffer shared between the producer and the consumer. The producer must have a condition associate with writing to the buffer stating that the buffer must not be full. The consumer must have a condition stating that the buffer cannot be empty. See the following example using the Ada language.

with Ada.Text_IO; use Ada.Text_IO;

procedure Main is
   type Order_Nums is range 1..10_000;
   Type Index is mod 10;
   type Buf_T is array(Index) of Order_Nums;

   protected Orders is
      entry Prepare(Order : in Order_Nums);
      entry Sell(Order : out Order_Nums);
   private
      Buffer  : Buf_T;
      P_Index : Index := Index'First;
      S_Index : Index := Index'First;
      Count   : Natural := 0;
   end Orders;

   protected body Orders is
      entry Prepare(Order : in Order_Nums) when Count < Index'Modulus is
      begin
         Buffer(P_Index) := Order;
         P_Index := P_Index + 1;
         Count := Count + 1;
      end Prepare;

      entry Sell(Order : out Order_Nums) when Count > 0 is
      begin
         Order := Buffer(S_Index);
         S_Index := S_Index + 1;
         Count := Count - 1;
      end Sell;
   end Orders;

   task Chef is
      Entry Stop;
   end Chef;

   task Seller is
      Entry Stop;
   end Seller;

   task body Chef is
      The_Order : Order_Nums := Order_Nums'First;
   begin
      loop
         select
            accept Stop;
            exit;
         else
            delay 1.0; -- one second
            Orders.Prepare(The_Order);
            Put_Line("Chef made order number " & The_Order'Image);
            The_Order := The_Order + 1;
            exit when The_Order = Order_Nums'Last;
         end select;
      end loop;
   end Chef;

   task body Seller is
      The_Order : Order_Nums;
   begin
      loop
         select
            accept Stop;
            exit;
         else
            delay 2.0; -- two seconds
            Orders.Sell(The_Order);
            Put_Line("Sold order number " & The_Order'Image);
         end select;
      end loop;
   end Seller;

begin
   delay 60.0; -- 60 seconds
   Chef.Stop;
   Seller.Stop;
end Main;

The shared buffer is named Orders. Orders contains a circular buffer of 10 Order_Nums. The index for the array containing the orders is declared as mod 10 which contains the values 0 through 9. Ada modular types exhibit wrap-around arithmetic, so incrementing past 9 wraps to 0. The Prepare entry has a boundary condition requiring Count < Index'Moduluswhich evaluates to Count < 10 in this instance. The Sell entry has a boundary condition Count < 0. The Chef task waits 1 second to produce a pizza, but waits until there is room in the buffer. As soon as there is room in the buffer Chef produces an order. Seller waits 2 seconds to consume an order. Each task terminates when its Stop entry is called. Main waits 60 seconds and then calls the Stop entries for each task. The output of the program is:

Chef made order number  1
Sold order number  1
Chef made order number  2
Chef made order number  3
Sold order number  2
Chef made order number  4
Chef made order number  5
Sold order number  3
Chef made order number  6
Chef made order number  7
Sold order number  4
Chef made order number  8
Chef made order number  9
Sold order number  5
Chef made order number  10
Chef made order number  11
Sold order number  6
Chef made order number  12
Chef made order number  13
Sold order number  7
Chef made order number  14
Chef made order number  15
Sold order number  8
Chef made order number  16
Chef made order number  17
Sold order number  9
Chef made order number  18
Chef made order number  19
Sold order number  10
Chef made order number  20
Sold order number  11
Chef made order number  21
Sold order number  12
Chef made order number  22
Chef made order number  23
Sold order number  13
Sold order number  14
Chef made order number  24
Sold order number  15
Chef made order number  25
Sold order number  16
Chef made order number  26
Chef made order number  27
Sold order number  17
Chef made order number  28
Sold order number  18
Chef made order number  29
Sold order number  19
Sold order number  20
Chef made order number  30
Sold order number  21
Chef made order number  31
Chef made order number  32
Sold order number  22
Sold order number  23
Chef made order number  33
Sold order number  24
Chef made order number  34
Sold order number  25
Chef made order number  35
Sold order number  26
Chef made order number  36
Chef made order number  37
Sold order number  27
Sold order number  28
Chef made order number  38
Sold order number  29
Chef made order number  39
Sold order number  30
Chef made order number  40
Sold order number  31