Jetty(v9.2.12) process logged too many open files warning due which log file size increased up to 377 GB and Jetty process was down.
2017-04-21 22:29:39.085:WARN:oejs.ServerConnector:qtp1921595561-113-acceptor-2@1a798c44-ServerConnector@5e3f861{HTTP/1.1}{0.0.0.0:18080}:
java.io.IOException: Too many open files
at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
at org.eclipse.jetty.server.ServerConnector.accept(ServerConnector.java:377)
at org.eclipse.jetty.server.AbstractConnector$Acceptor.run(AbstractConnector.java:500)
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
at java.lang.Thread.run(Thread.java:745)
When this issue occurred Jetty process hold 4311 total open files. PFB result of lsof command. Maximum connection were created of sock, unix & FIFO type. Can someone pls help me to understand why sock, unix & FIFO type connections Jetty creates.
Conn Type - Count CHR - 6 DIR - 2 FIFO - 1023 IPv4 - 53 REG - 925 sock - 554 unix - 1748 Total - 4311
Result of ulimit -a
core file size (blocks, -c) unlimited data seg size (kbytes, -d) unlimited scheduling priority (-e) 0 file size (blocks, -f) unlimited pending signals (-i) 2066962 max locked memory (kbytes, -l) 64 max memory size (kbytes, -m) unlimited open files (-n) 1024 pipe size (512 bytes, -p) 8 POSIX message queues (bytes, -q) 819200 real-time priority (-r) 0 stack size (kbytes, -s) 10240 cpu time (seconds, -t) unlimited max user processes (-u) 2066962 virtual memory (kbytes, -v) unlimited file locks (-x) unlimited
