0

We have a client and server communicating each other with grpc. Previously the server was running on Windows Server, and the client running on Linux or MacOS. Everything works perfectly until we migrate the server from Windows Server to a docker container.

Then we observed some weird tcp broken when we send a large amount of request from client to server.

Then we dig into the grpc arena and run our client and server with GRPC_VERBOSITY=info and GRPC_TRACE=tcp. Then we found that the disconnection was caused from the server side, with error message below:

tcp_custom.cc:218] write complete on 029FCC20: error={"created":"@1594210168.896000000","description":"TCP Write failed","file":"d:\a\grpc-node\grpc-node\packages\grpc-native-core\deps\grpc\src\core\lib\iomgr\tcp_uv.cc","file_line":72,"grpc_status":14,"os_error":"message too long"}

So my question is what does the os_error: message too long really means? What is the next step for me to investigate?

Linked issue

WindMemory
  • 570
  • 1
  • 4
  • 16

1 Answers1

1

The string "message too long" is the error message for the error code UV_EMSGSIZE, which corresponds to the Linux error code EMSGSIZE. That appears to generally mean that gRPC is trying to write a buffer that is too large to the socket, but I'm not sure what exactly could trigger that.

murgatroid99
  • 19,007
  • 10
  • 60
  • 95
  • Thanks for your answer! Then do you know where can I find the limitation for the socket message size? – WindMemory Jul 12 '20 at 02:15
  • I don't really know. https://stackoverflow.com/questions/57214403/emsgsize-when-trying-to-send-data-on-raw-ip-packet might help, or at least be a good starting point. – murgatroid99 Jul 12 '20 at 02:45
  • I saw that question after you point the error to `EMSGSIZE`. I haven't find the root cause yet, but at least I have another thing to investigate, thanks very much! – WindMemory Jul 13 '20 at 03:25
  • I saw that question after you point the error to `EMSGSIZE`. I haven't find the root cause yet, but at least I have another thing to investigate, thanks very much! – WindMemory Jul 13 '20 at 03:26