gRPC streaming in the separate process get stuck

Question

I'm trying to create a client for testing my grpc server. In my grpc server I have a rpc NotificationStreaming() which streams a notification (unary-stream). Also I have bunch of synchronous rpc methods (unary-unary).

In the main() below first I establish a connection for the streaming in the separate process and then I perform unary-unary rpc requests sequentially. After each unary-unary rpc request I receive notifications via NOTIFICATION_QUEUE. The streaming stays empty until I call the fist unary-unary method create_project(stub), so I'm expecting to receive a first notification during this method.

The problem is that if I remove sleep(5) my program stuck at this line.

Please give me any ideas about how to use more wise way?

def _notification_stream(notification_queue):
    with grpc.insecure_channel(settings.GRPC_PORT) as channel:
        stub = main_pb2_grpc.MyAPIStub(channel)
        try:
            response_stream = stub.NotificationStreaming(Empty())
            for notification in response_stream:
                r = json_format.MessageToDict(notification, preserving_proto_field_name=True,
                                              including_default_value_fields=True)
                notification_queue.put(r['message'])
        except grpc.RpcError as e:
            misc.log(f"ERROR notification stream: {e}")


def notification_streaming(notification_queue):
    _process = mp.Process(target=_notification_stream, daemon=True, kwargs={"notification_queue": notification_queue})
    _process.start()
    return _process.pid



def main():
    NOTIFICATION_QUEUE = mp.Queue()
    # start listening to the notification stream
    notification_streaming(NOTIFICATION_QUEUE)
    sleep(5)
    with grpc.insecure_channel(settings.GRPC_PORT) as channel:
        stub = main_pb2_grpc.MyAPIStub(channel)
        create_project(stub)
        while not NOTIFICATION_QUEUE.empty():
            misc.log(f"\tnotification: {NOTIFICATION_QUEUE.get(block=True)}")
        close_project(stub)
        while not NOTIFICATION_QUEUE.empty():
            misc.log(f"\tnotification: {NOTIFICATION_QUEUE.get(block=True)}")
        load_project(stub)
        while not NOTIFICATION_QUEUE.empty():
            misc.log(f"\tnotification: {NOTIFICATION_QUEUE.get(block=True)}")
        save_project(stub)
        while not NOTIFICATION_QUEUE.empty():
            misc.log(f"\tnotification: {NOTIFICATION_QUEUE.get(block=True)}")
        ...


if __name__ == '__main__':
    main()

score 0 · Answer 1 · answered May 24 '22 at 12:30

The idea is to make sure that the channel is established before creating a new channel. For this reason channel.subscribe(wait_for_ready_partial, try_to_connect=True) for checking the state of ChannelConnectivity and continue only when grpc.ChannelConnectivity.READY

NOTIFICATION_QUEUE = mp.Queue()

def _notification_stream(notification_queue):
    def wait_for_ready(channel_connectivity, _notification_queue=None):
        if channel_connectivity is grpc.ChannelConnectivity.READY:
            _notification_queue.put(settings.STOP_WORD)

    with grpc.insecure_channel(settings.GRPC_PORT) as channel:
        wait_for_ready_partial = functools.partial(wait_for_ready, _notification_queue=notification_queue)
        channel.subscribe(wait_for_ready_partial, try_to_connect=True)
        stub = main_pb2_grpc.MyAPIStub(channel)
        response_stream = stub.NotificationStreaming(Empty())
        for notification in response_stream:
            r = json_format.MessageToDict(notification, preserving_proto_field_name=True,
                                          including_default_value_fields=True)
            notification_queue.put(r['message'])


def notification_streaming(notification_queue):
    _process = mp.Process(target=_notification_stream, daemon=True, kwargs={"notification_queue": notification_queue})
    _process.start()
    while True:
        if notification_queue.get(block=True) == settings.STOP_WORD:
            break

def main():
    # establish the channel for Notification streaming (unary-stream)
    notification_streaming(NOTIFICATION_QUEUE)
    # the main grpc unary-unary intecations
    with grpc.insecure_channel(settings.GRPC_PORT) as channel:
        stub = main_pb2_grpc.MyAPIStub(channel)
        create_project(stub)
        close_project(stub)
        load_project(stub)
        ...


if __name__ == '__main__':
    main()

gRPC streaming in the separate process get stuck

1 Answers1