2

Unable to timeout a grpc connection from server side. It is possible that client establishes a connection but kept on hold/sleep which is resulting in grpc server connection to hang. Is there a way at server side to disconnect the connection after a certain time or set the timeout?

We tried disconnecting the connection from client side but unable to do so from server side. In this link Problem with gRPC setup. Getting an intermittent RPC unavailable error, Angad says that it is possible but unable to define those parameters in python.

My code snippet:

def serve():
server = grpc.server(thread_pool=futures.ThreadPoolExecutor(max_workers=2), maximum_concurrent_rpcs=None, options=(('grpc.so_reuseport', 1),('grpc.GRPC_ARG_KEEPALIVE_TIME_MS', 1000)))
stt_pb2_grpc.add_ListenerServicer_to_server(Listener(), server)
server.add_insecure_port("localhost:50051")
print("Server starting in port "+str(50051))
server.start()
try:
    while True:
        time.sleep(60 * 60 * 24)
except KeyboardInterrupt:
    server.stop(0)
if __name__ == '__main__':
serve()

I expect the connection should be timed out from grpc server side too in python.

Mahesh
  • 171
  • 3
  • 8

1 Answers1

1

In short, you may find context.abort(...) useful, see API reference. Timeout a server handler is not supported by the underlying C-Core API of gRPC Python. So, you have to implement your own timeout mechanism in Python.

You can try out some solution from other StackOverflow questions.

Or use a simple-but-big-overhead extra threads to abort the connection after certain length of time. It might look like this:

_DEFAULT_TIME_LIMIT_S = 5

class FooServer(FooServicer):

  def RPCWithTimeLimit(self, request, context):
    rpc_ended = threading.Condition()
    work_finished = threading.Event()

    def wrapper(...):
      YOUR_ACTUAL_WORK(...)
      work_finished.set()
      rpc_ended.notify_all()

    def timer():
      time.sleep(_DEFAULT_TIME_LIMIT_S)
      rpc_ended.notify_all()

    work_thread = threading.Thread(target=wrapper, ...)
    work_thread.daemon = True
    work_thread.start()

    timer_thread = threading.Thread(target=timer)
    timer_thread.daemon = True
    timer_thread.start()

    rpc_ended.wait()
    if work_finished.is_set():
      return NORMAL_RESPONSE
    else:
      context.abort(grpc.StatusCode.DEADLINE_EXCEEDED, 'RPC Time Out!')
Lidi Zheng
  • 1,801
  • 8
  • 13