7

How would I generate streaming response values for an RPC from outside the handler? (specifically, from a IObservable) I'm currently doing the following, but this is creating cross-thread issues because AnRxObservable is shared between RPC handlers...

public override Task GetTicker(RequestProto request, ServerCallContext context)
{
    var subscription = AnRxObservable.Subscribe(value =>
    {
        responseStream.WriteAsync(new ResponseProto
        {
            Value = value
        });
    });

    // Wait for the RPC to be canceled (my extension method
    // that returns a task that completes when the CancellationToken
    // is cancelled)
    await context.CancellationToken.WhenCancelled();

    // Dispose of the buffered stream
    bufferedStream.Dispose();

    // Dispose subscriber (tells rx that we aren't subscribed anymore)
    subscription.Dispose();

    return Task.FromResult(1);
}

This code doesn't feel right... but I can't see any other way of streaming RPC responses from a shared source created outside the RPC handler.

Warrick
  • 1,623
  • 1
  • 17
  • 20
  • What do you mean by "causing cross-thread issues"? – Jan Tattermusch Aug 04 '17 at 07:30
  • The thread that the observable pushed data from wasn't the thread that GetTicker was called on (from inside GRPC). As an example, the observable pushes values from thread id 1 always, but GRPC will call GetTicker on a different thread for each request (from the thread pool I belive). The issue is when there are two concurrent GetTicker RPC requests... the streams would stop being received by the client unpredictably. TLDR; is GRPC thread safe... it appears not, but I couldn't find any evidence to back this up. – Warrick Aug 04 '17 at 08:52

1 Answers1

7

Generally speaking, when you are trying to convert from push model (IObservable) into pull model (enumerating the responses to write and writing them), you need an intermediate buffer for the message - e.g. a blockingQueue. The handler body can then be an async loop that tries to fetch the next message for the queue (preferably in an async fashion) and writes it to the responseStream.

Also, be aware that gRPC API only allows you to have 1 in-flight response at any given time - and your snippet doesn't respect that. So you need to await the WriteAsync() before starting another write (an that's another reason why you need an intermediate queue).

This link might be useful in explaining the push vs pull paradigms: When to use IEnumerable vs IObservable?

Jan Tattermusch
  • 1,515
  • 9
  • 9
  • I actually did encounter the 1 in-flight issue, and implemented a buffer (https://gist.github.com/warrickw/f1795e188aeda5b59eae3e0ad786245a)... which in my case overflowed quite a lot. Does GRPC wait for a message acknoldgement (which could be 200ms later) before sending the next value - effectively imposing a latency-based bandwidth bottleneck? – Warrick Aug 04 '17 at 08:48
  • 1
    gRPC waits with confirming the write operation until it sends out the message on the wire (once TCP/HTTP flow control windows allows it) - this is necessary to ensure flow control (without it, you'd flood slower peers easily). – Jan Tattermusch Aug 10 '17 at 12:01