Netty - EventLoop Queue Monitoring

Question

I am using Netty server for a Spring boot application. Is there anyway to monitor the Netty server queue size so that we will come to know if the queue is full and server is not able to accept any new request? Also, Is there any logging by netty server if the queue is full or unable to accept a new request?

Answer 1

Netty does not have any logging for that purpose but I implemented a way to find pending tasks and put some logs according to your question. here is a sample log from my local

you can find all code here https://github.com/ozkanpakdil/spring-examples/tree/master/reactive-netty-check-connection-queue

About code which is very explanatory from itself but NettyConfigure is actually doing the netty configuration in spring boot env. at https://github.com/ozkanpakdil/spring-examples/blob/master/reactive-netty-check-connection-queue/src/main/java/com/mascix/reactivenettycheckconnectionqueue/NettyConfigure.java#L46 you can see "how many pending tasks" in the queue. DiscardServerHandler may help you how to discard if the limit is full. You can use jmeter for the test here is the jmeter file https://github.com/ozkanpakdil/spring-examples/blob/master/reactive-netty-check-connection-queue/PerformanceTestPlanMemoryThread.jmx

if you want to handle netty limit you can do it like the code below

@Override
public void channelActive(ChannelHandlerContext ctx) throws Exception {
    totalConnectionCount.incrementAndGet();
    if (ctx.channel().isWritable() == false) { // means we hit the max limit of netty
        System.out.println("I suggest we should restart or put a new server to our pool :)");
    }
    super.channelActive(ctx);
}

You should check https://stackoverflow.com/a/49823055/175554 for handling the limits and here is another explanation about "isWritable" https://stackoverflow.com/a/44564482/175554

One more extra, I put actuators in the place http://localhost:8080/actuator/metrics/http.server.requests is nice to check too.