I have a server application that can return a huge response (>10gb) via HTTP. The response is dynamically generated and sent in chunks to the client. The client (curl
) could be slow in consuming the content generated by the server, this causes the server to stop and wait. The problem is that the server then never resumes.
I'm using EpollEventLoopGroup
.
Here is what seems to happen:
- The server spits out data really fast (using
writeAndFlush()
). - The client cannot keep up reading the data (I saw several posts about Netty and slow clients)
- The server stops sending data by setting
setFlag(Native.EPOLLOUT);
: https://github.com/netty/netty/blob/4.1/transport-classes-epoll/src/main/java/io/netty/channel/epoll/AbstractEpollStreamChannel.java#L447-L449 - The server stops sending data because calls to flush are actually unsuccessful, see this check: https://github.com/netty/netty/blob/4.1/transport-classes-epoll/src/main/java/io/netty/channel/epoll/AbstractEpollChannel.java#L551-L559 (note this checks for the same flag set above
!isFlagSet(Native.EPOLLOUT)
) - For some reason the server never goes back into a writing state (
clearFlag(Native.EPOLLOUT)
). - Since no data is flushed to the client the
ChannelOutboundBuffer
keeps growing, as the server keeps producing data, until I get OOM (Netty Direct Buffer). - If after the
writeAndFlush()
I check forchannel.isWritable()
and stop until the channel is writable again, it will wait forever, because the channel will never be writable again.
My question is, why does the server never goes back into writing to the client?