0

Let us say, a gRPC client makes two requests R1 and R2 to gRPC server, one after the other (assume without any significant time gap, i.e R2 is made when R1 is still not served). Also, assume that R1 takes much more time than R2. In this case, should I expect R2's response first as it takes less time or should I expect R1's response first as this request is made prior to R2? What will happen and why? As far as what I have observed, I think requests are served in FCFS fashion, so, R1's response will be received by the client first and then R2's, but I am not sure.

honor
  • 7,378
  • 10
  • 48
  • 76
sv2311
  • 65
  • 2
  • 10

2 Answers2

2

Theoretically nothing discourages server and client process gRPC requests in parallel. GRPC connection is made over HTTP/2 one that can handle multiple requests at once. So yes - if server doesn't use some specific synchronization or limitation mechanisms then requests would be processes with overlapping. If server resources or policy doesn't allow it then they should be processed one by one. Also I can add than request can have a Timeout after which it would be cancelled. So long wait can lead to cancellation and non-processing at all.

Eugene Lisitsky
  • 12,113
  • 5
  • 38
  • 59
-2

All requests should be processed in parallel. The gRPC architecture for the Java implementation for example, it is divided into 2 "parts":

  • The event loop runs in a thread work group - It is similar to what we have to reactive implementations. One thread per core to handle the incoming requests.

  • The request processing is done in a dedicated thread which will be created using the CachedThreadPool system by default.

For single-thread languages like Javascript, I am not sure how they are doing it, but I would guess it is done in the same thread and therefore it would end up queuing the requests.

Alexsandro Souza
  • 806
  • 7
  • 14
  • The JavaScript gRPC interface can handle multiple simultaneous requests. It uses a future/promise model so requests are handled asynchronously and a callback is invoked when the reply (or error) is received, and there can be multiple outstanding requests which are not necessarily completed in order. – Tim Mar 19 '22 at 09:08
  • Like HTTP requests. gRPC will handle requests in a single thread as I said. future/promise happens in a single thread. Good luck avoiding blocking the IO operations. – Alexsandro Souza Mar 20 '22 at 22:00
  • Using a single thread does not mean that requests are queued, or that IO blocks. You can service many non-blocking streams in parallel from a single thread. – Tim Mar 20 '22 at 23:49