0

I am maxing out my CPU on my gRPC client doing unary RPC. I wonder if it makes any sense to try replacing unary RPC with a bidirectional streaming RPC (or set of them?) that basically lasts throughout the life of the application? I can't tell if bidirectional streaming RPC is intended for standard 1 request/1 response communication like this. The motivation would be to avoid creating new TCP connections.

vmayer
  • 985
  • 2
  • 9
  • 18

2 Answers2

3

I can't tell if bidirectional streaming RPC is intended for standard 1 request/1 response communication

If you are going to use a request-reply message pattern, just use unary request-reply (RPC). It is designed for that pattern and semantics for e.g. retry is well known.

The motivation would be to avoid creating new TCP connections.

gRPC uses HTTP/2, so all unary RPC requests already use the same TCP connection - since HTTP/2 multiplexes all requests over the same TCP connection.

I am maxing out my CPU on my gRPC client doing unary RPC.

This sounds a bit rare. Can you change your communication pattern, e.g. stream multiple requests before waiting for a response? Alternative batch data and send bigger requests more seldom? Or can you elaborate more about your message pattern?

Jonas
  • 121,568
  • 97
  • 310
  • 388
  • Thanks for the quick response. As far as using the same TCP connection, I guess I'm a bit confused because it seems my tcpdump only shows TCP streams lasting momentarily. The client call is auto-generated code (in Go) which calls ClientConn.Invoke(), which under the hood calls newClientStream(), SendMsg(), and then ReceiveMsg() here: https://github.com/grpc/grpc-go/blob/master/call.go. Is there something I'm missing? – vmayer Jan 11 '20 at 04:53
  • As far as maxing out the CPU, I think I should try building a separate application which only performs this grpc and does nothing else before I press this issue too much. – vmayer Jan 11 '20 at 04:54
  • 1
    That should be fine. `ClientConn` represents the TCP connection https://github.com/grpc/grpc-go/blob/master/Documentation/concurrency.md and you can create multiple streams within the same ClientCon. If code executes so that your ClientCon comes out of scope, it will be garbage collected and your TCP connection is terminated. You have to carefully take care of your connection if you want it to stay open. – Jonas Jan 11 '20 at 17:49
  • Oh, I see, it's because I keep calling grpc.Dial() over and over that I keep ending up with a new ClientConn. I don't need to be doing that I guess. – vmayer Jan 11 '20 at 18:12
  • Do you think it would make sense to use many ClientConns (that don't go out of scope) with a Balancer in front? (hope I'm saying that right) – vmayer Jan 11 '20 at 18:14
  • I think it only make sense to use one per server... since it is multiplexing messages in possibly a large number of streams. – Jonas Jan 11 '20 at 18:27
  • To clarify that last question, those ClientConns would all be going to the same load balancer (K8 service) sitting in front of the instances of my application. Just thinking it would be more performant to have many of those? – vmayer Jan 11 '20 at 18:28
  • Oh, not more performant you think to have multiple TCP connections? (I have a large rate of messages flowing) – vmayer Jan 11 '20 at 18:30
0

Like @Jonas suggested, using a bidirectional stream just for higher throughput would be a bad idea.

Google’s gRPC team advises against it, but nevertheless, few seem to argue that theoretically, streams should have lower overhead. But that does not seem to be true.

Probably because streams ensure the messages are delivered in the order they were sent, thus creating some kind of bottleneck when there are concurrent messages.

For lower concurrent requests, both have comparable latencies. However, for higher loads, unary calls are much more performant.

A detailed analysis here: https://nshnt.medium.com/using-grpc-streams-for-unary-calls-cd64a1638c8a.

Nishant
  • 4,659
  • 2
  • 27
  • 43