1

My server can handle 10000qps, but when it comes 11000qps, it's overloading. That will cause a out of memory eventually.

I'm wondering how grpc-java deal with server overloading. I've searched for a long time, no configurable version found (parameters like max-in-flight-messages).

There's only a manual version in grpc-java examples.

Would you give me some advice? Thanks in advance.

Brutal_JL
  • 2,839
  • 2
  • 21
  • 27

2 Answers2

1

Take a look at maxConcurrentCallsPerConnection[1]. You need to use the NettyServerBuilder explicitly. But you are most likey using that one, under the hood anyway. (This might not be enough if you dont control the (number of) clients)

You could also look at https://github.com/Netflix/concurrency-limits. It will most likely solve your concurrency issues.

[1] https://grpc.github.io/grpc-java/javadoc/io/grpc/netty/NettyServerBuilder.html#maxConcurrentCallsPerConnection-int-

Schildmeijer
  • 20,702
  • 12
  • 62
  • 79
  • Thanks, `maxConcurrentCallsPerConnection ` works well in unary call. Btw, why the http2 connection flow control window not working as a default choice? – Brutal_JL Nov 25 '19 at 07:11
  • The flow control does not apply for unary calls because it only sends one message per h2 stream by grpc library. Using maxConcurrentCallsPerConnection and responding slowly is the right solution for unary calls to avoid server overloading. – user675693 Dec 30 '19 at 18:56
  • I can highly recommend the Netflix concurrency limits. Applying it is easy and it works great in making the system responsive (by dropping excess requests). – George Leung Mar 14 '21 at 17:41
0

Another option is grpc reactive streams : https://github.com/salesforce/reactive-grpc

It will provide better back pressure and flow control.

Nicholas
  • 15,916
  • 4
  • 42
  • 66