Akka stream - limiting Flow rate without introducing delay

Question

I'm working with Akka (version 2.4.17) to build an observation Flow in Java (let's say of elements of type <T> to stay generic).

My requirement is that this Flow should be customizable to deliver a maximum number of observations per unit of time as soon as they arrive. For instance, it should be able to deliver at most 2 observations per minute (the first that arrive, the rest can be dropped).

I looked very closely to the Akka documentation, and in particular this page which details the built-in stages and their semantics.

So far, I tried the following approaches.

• With throttle and shaping() mode (to not close the stream when the limit is exceeded):

    Flow.of(T.class)
         .throttle(2, 
                   new FiniteDuration(1, TimeUnit.MINUTES), 
                   0, 
                   ThrottleMode.shaping())
  

• With groupedWith and an intermediary custom method:

  final int nbObsMax = 2;
  
  Flow.of(T.class)
      .groupedWithin(Integer.MAX_VALUE, new FiniteDuration(1, TimeUnit.MINUTES))
      .map(list -> {
           List<T> listToTransfer = new ArrayList<>();
           for (int i = list.size()-nbObsMax ; i>0 && i<list.size() ; i++) {
               listToTransfer.add(new T(list.get(i)));
           }
           return listToTransfer;
      })
      .mapConcat(elem -> elem)  // Splitting List<T> in a Flow of T objects
  

Previous approaches give me the correct number of observations per unit of time but these observations are retained and only delivered at the end of the time window (and therefore there is an additional delay).

To give a more concrete example, if the following observations arrives into my Flow:

[Obs1 t=0s] [Obs2 t=45s] [Obs3 t=47s] [Obs4 t=121s] [Obs5 t=122s]

It should only output the following ones as soon as they arrive (processing time can be neglected here):

Window 1: [Obs1 t~0s] [Obs2 t~45s] Window 2: [Obs4 t~121s] [Obs5 t~122s]

Any help will be appreciated, thanks for reading my first StackOverflow post ;)

Answer 1

I cannot think of a solution out of the box that does what you want. Throttle will emit in a steady stream because of how it is implemented with the bucket model, rather than having a permitted lease at the start of every time period.

To get the exact behavior you are after you would have to create your own custom rate-limit stage (which might not be that hard). You can find the docs on how to create custom stages here: http://doc.akka.io/docs/akka/2.5.0/java/stream/stream-customize.html#custom-linear-processing-stages-using-graphstage

One design that could work is having an allowance counter saying how many elements that can be emitted that you reset every interval, for every incoming element you subtract one from the counter and emit, when the allowance used up you keep pulling upstream but discard the elements rather than emit them. Using TimerGraphStageLogic for GraphStageLogic allows you to set a timed callback that can reset the allowance.

Answer 2

I think this is exactly what you need: http://doc.akka.io/docs/akka/2.5.0/java/stream/stream-cookbook.html#Globally_limiting_the_rate_of_a_set_of_streams

Answer 3

Thanks to the answer of @johanandren, I've successfully implemented a custom time-based GraphStage that meets my requirements.

I post the code below, if anyone is interested:

import akka.stream.Attributes;
import akka.stream.FlowShape;
import akka.stream.Inlet;
import akka.stream.Outlet;
import akka.stream.stage.*;
import scala.concurrent.duration.FiniteDuration;

public class CustomThrottleGraphStage<A> extends GraphStage<FlowShape<A, A>> {

    private final FiniteDuration silencePeriod;
    private int nbElemsMax;

    public CustomThrottleGraphStage(int nbElemsMax, FiniteDuration silencePeriod) {
        this.silencePeriod = silencePeriod;
        this.nbElemsMax = nbElemsMax;
    }

    public final Inlet<A> in = Inlet.create("TimedGate.in");
    public final Outlet<A> out = Outlet.create("TimedGate.out");

    private final FlowShape<A, A> shape = FlowShape.of(in, out);
    @Override
    public FlowShape<A, A> shape() {
        return shape;
    }

    @Override
    public GraphStageLogic createLogic(Attributes inheritedAttributes) {
        return new TimerGraphStageLogic(shape) {

            private boolean open = false;
            private int countElements = 0;

            {
                setHandler(in, new AbstractInHandler() {
                    @Override
                    public void onPush() throws Exception {
                        A elem = grab(in);
                        if (open || countElements >= nbElemsMax) {
                            pull(in);  // we drop all incoming observations since the rate limit has been reached
                        }
                        else {
                            if (countElements == 0) { // we schedule the next instant to reset the observation counter
                                scheduleOnce("resetCounter", silencePeriod);
                            }
                            push(out, elem); // we forward the incoming observation
                            countElements += 1; // we increment the counter
                        }
                    }
                });
                setHandler(out, new AbstractOutHandler() {
                    @Override
                    public void onPull() throws Exception {
                        pull(in);
                    }
                });
            }

            @Override
            public void onTimer(Object key) {
                if (key.equals("resetCounter")) {
                    open = false;
                    countElements = 0;
                }
            }
        };
    }
}