0

We have a Java Spring web application with Log4j2 configured with RollingFile active running on a Tomcat server farm with NFS storage configured.

Catalina out log is writing a lot of execptions java.io.IOException: Obsolete file handle apparently when Log4j is rolling the file (compressing the log file an generating a new log)

Let´s say we have app.log with descriptor 521 and this is the file descriptor all the servers of the farm has. webserver1.com is writing in app.log and detects it has to do a file rolling, compress the log and generates a new app.log and this time its file descriptor is 853. All the other webserver.com has during a time the file descriptor 521 and fails to write to the log file and produces the exception.

Is there any option in log4j2 that empty the app.log file instead of creating a new one so the app.log mantains always the same file descriptor? Or any other suggestions?

Thanks.

Edited: The exact error from catalina logs is this

2022-04-27 08:27:23,533 ajp-nio-127.0.0.1-8009-exec-4 ERROR An exception occurred processing Appender RollingFileAppender org.apache.logging.log4j.core.appender.AppenderLoggingException: Error writing to stream /webges/app<xxxx>2021/logs/app<xxxx>2021.log
        at org.apache.logging.log4j.core.appender.OutputStreamManager.writeToDestination(OutputStreamManager.java:252)
        at org.apache.logging.log4j.core.appender.FileManager.writeToDestination(FileManager.java:277)
        at org.apache.logging.log4j.core.appender.rolling.RollingFileManager.writeToDestination(RollingFileManager.java:275)
        at org.apache.logging.log4j.core.appender.OutputStreamManager.flushBuffer(OutputStreamManager.java:283)
        at org.apache.logging.log4j.core.appender.OutputStreamManager.flush(OutputStreamManager.java:294)
        at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.directEncodeEvent(AbstractOutputStreamAppender.java:199)
        at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.tryAppend(AbstractOutputStreamAppender.java:190)
        at org.apache.logging.log4j.core.appender.AbstractOutputStreamAppender.append(AbstractOutputStreamAppender.java:181)
        at org.apache.logging.log4j.core.appender.RollingFileAppender.append(RollingFileAppender.java:312)
        at org.apache.logging.log4j.core.config.AppenderControl.tryCallAppender(AppenderControl.java:161)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppender0(AppenderControl.java:134)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppenderPreventRecursion(AppenderControl.java:125)
        at org.apache.logging.log4j.core.config.AppenderControl.callAppender(AppenderControl.java:89)
        at org.apache.logging.log4j.core.config.LoggerConfig.callAppenders(LoggerConfig.java:542)
        at org.apache.logging.log4j.core.config.LoggerConfig.processLogEvent(LoggerConfig.java:500)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:483)
        at org.apache.logging.log4j.core.config.LoggerConfig.log(LoggerConfig.java:417)
        at org.apache.logging.log4j.core.config.AwaitCompletionReliabilityStrategy.log(AwaitCompletionReliabilityStrategy.java:82)
        at org.apache.logging.log4j.core.Logger.log(Logger.java:161)
        at org.apache.logging.log4j.spi.AbstractLogger.tryLogMessage(AbstractLogger.java:2205)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessageTrackRecursion(AbstractLogger.java:2159)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessageSafely(AbstractLogger.java:2142)
        at org.apache.logging.log4j.spi.AbstractLogger.logMessage(AbstractLogger.java:1994)
        at org.apache.logging.log4j.spi.AbstractLogger.logIfEnabled(AbstractLogger.java:1852)
        at org.apache.logging.log4j.jcl.Log4jLog.debug(Log4jLog.java:81)
        at org.springframework.core.log.LogFormatUtils.traceDebug(LogFormatUtils.java:91)
        at org.springframework.web.servlet.DispatcherServlet.logRequest(DispatcherServlet.java:979)
        at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:925)
        at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:1006)
        at org.springframework.web.servlet.FrameworkServlet.doGet(FrameworkServlet.java:898)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:655)
        at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:883)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:764)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.logging.log4j.web.Log4jServletFilter.doFilter(Log4jServletFilter.java:71)
        at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
        at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
        at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:196)
        at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
        at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
        at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:135)
        at psiprobe.Tomcat85AgentValve.invoke(Tomcat85AgentValve.java:35)
        at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
        at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:698)
        at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
        at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:366)
        at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:526)
        at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
        at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:847)
        at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1680)
        at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1191)
        at org.apache.tomcat.util.threads.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:659)
        at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
        at java.lang.Thread.run(Thread.java:750)
Caused by: java.io.IOException: Identificador de archivos obsoletos
        at java.io.FileOutputStream.writeBytes(Native Method)
        at java.io.FileOutputStream.write(FileOutputStream.java:326)
        at org.apache.logging.log4j.core.appender.OutputStreamManager.writeToDestination(OutputStreamManager.java:250)
        ... 59 more 
  • Can you copy/paste the exact error from the logs? Is it really "Obsolete file handle" or rather "Stale file handle"? You might have multiple instances of your application that try to rotate the same file at the same time. – Piotr P. Karwasz May 14 '22 at 08:11
  • 1
    Do you really have every server in the farm writing to the same single file hosted on NFS storage? That sounds like it could never work properly. – Christopher Schultz May 14 '22 at 13:18

0 Answers0