1

I'm working on a program which will asynchronously load large amounts of data from a server via gRPC (both ends use protobuf-net.Grpc).

While I can simply throw large amounts of data at the client via IAsyncEnumerable, I sometimes want to prioritize sending certain parts earlier (where the priorities are determined on the fly, and not know at the start, kind of like sending a video stream and skipping ahead).

If I were to send data and wait for a response every time I'd be leaving a lot of bandwidth unused. The alternative would be throwing tons of data at the client, which could cause network congestion and delay prioritized packets by an indefinite amount.

Can I somehow use HTTPS/2s / TCPs flow/congestion control for myself here? Or will I need to implement a basic flow/congestion control system on top of gRPC?

To be slightly more precise: I'd like to send as much data as possible without filling up any internal buffers, causing delays down the line.

blenderfreaky
  • 738
  • 7
  • 26
  • I didn't know http/2 had that sort of thing. Cool. Is there a reference link for it? – mtmk Dec 16 '21 at 17:32
  • 1
    At first look kestrel implementation seems to be internal, first thought; not sure if some middleware api can control that https://github.com/dotnet/aspnetcore/tree/main/src/Servers/Kestrel/Core/src/Internal/Http2/FlowControl – mtmk Dec 16 '21 at 17:43
  • 1
    @ziya https://en.wikipedia.org/wiki/TCP_congestion_control congestion control is handled by TCP, flow control by HTTP/2: https://medium.com/coderscorner/http-2-flow-control-77e54f7fd518 (or ietf https://tools.ietf.org/html/rfc7540) – blenderfreaky Dec 16 '21 at 17:48
  • thanks @blenderfreaky .. Did you make any progress? I was wondering if you could write a middleware to make it work, but I don't know enough about middleware. Really interesting question btw +1 – mtmk Dec 17 '21 at 19:38
  • Not yet - currently it seems like it'd be easiest to simply implement my own. Another approach might be implementing a custom Stream/IAsyncEnumerable (so not with yield), which generates data when Read is called, assuming there are no internal buffers (since you'd probably just pass a BufferedStream yourself usually). ASP.NET should allow sending streams, so maybe something similar is possible here with protobuf-net – blenderfreaky Dec 18 '21 at 16:36

0 Answers0