JBoss/Wildfly - Safe and efficient log filtering - Multiple handlers logging to the same file

Question

What is the safest and most efficient way to apply filter expressions to log messages that always occur with the same category?

I have over 100 applications in one container logging to the same file. The messages I want to handle are very specific. Applying a single complex filter-spec to every message seems like a lot of overhead, so I decided create individual loggers for each category with a message I want to filter. Each logger gets its own handler with its own filter-spec. That way I would only apply the filtering logic to the smallest number of log messages possible.

Here are the loggers in standalone.xml:

<root-logger>
    <level name="INFO"/>
    <handlers>
        <handler name="CONSOLE"/>
        <handler name="FILE"/>
    </handlers>
</root-logger>
<logger category="org.hibernate.engine.jdbc.spi.SqlExceptionHelper" use-parent-handlers="false">
    <level name="INFO"/>
    <handlers>
        <handler name="CONSOLE"/>
        <handler name="SQL_EXECUTION_HELPER_FILE"/>
    </handlers>
</logger>
<logger category="org.jboss.as.ejb3.invocation" use-parent-handlers="false">
    <level name="INFO"/>
    <handlers>
        <handler name="CONSOLE"/>
        <handler name="EJB3_INVOCATION_FILE"/>
    </handlers>
</logger>

Here are the file handlers:

<periodic-rotating-file-handler name="FILE" autoflush="true">
    <formatter>
        <named-formatter name="PATTERN"/>
    </formatter>
    <file relative-to="jboss.server.log.dir" path="server.log"/>
    <suffix value=".yyyy-MM-dd"/>
    <append value="true"/>
</periodic-rotating-file-handler>
<periodic-rotating-file-handler name="SQL_EXECUTION_HELPER_FILE">
    <filter-spec value="any(  not(match(&quot;ERROR: duplicate key value violates unique constraint \&quot;option_option_expiry_id_option_type_id_strike_price_key\&quot;&quot;)),  all(  match(&quot;ERROR: duplicate key value violates unique constraint \&quot;option_option_expiry_id_option_type_id_strike_price_key\&quot;&quot;),  levelChange(WARN)  )  )"/>
    <formatter>
        <named-formatter name="PATTERN"/>
    </formatter>
    <file relative-to="jboss.server.log.dir" path="server.log"/>
    <suffix value=".yyyy-MM-dd"/>
    <append value="true"/>
</periodic-rotating-file-handler>
<periodic-rotating-file-handler name="EJB3_INVOCATION_FILE">
    <filter-spec value="any(  not(match(&quot;EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert&quot;)),  all(  match(&quot;EJB Invocation failed on component OptionProductManagementDataService for method public void com.nodalexchange.optionproductmanagement.OptionProductManagementDataService.insert&quot;),  levelChange(WARN)  )  )"/>
    <formatter>
        <named-formatter name="PATTERN"/>
    </formatter>
    <file relative-to="jboss.server.log.dir" path="server.log"/>
    <suffix value=".yyyy-MM-dd"/>
    <append value="true"/>
</periodic-rotating-file-handler>

I don't understand how these file handlers work under the hood, for example whether they share a file descriptor or each open their own. This has always worked fine for me except there was one incident where I was only getting the messages from a single handler and nothing else, which leads me to believe it is unsafe to use multiple handlers, but I have not been able to reproduce this.

Is there a risk of the files becoming corrupted with many handlers writing to the same file?
Is there a better way?

score 1 · Accepted Answer · answered Sep 21 '19 at 00:14

1

I would definitely advise against using multiple handlers writing to the same file. There could definitely be issues with that.

Since you're defining specific loggers you'd like to see filter you should put the filter on those loggers instead.

answered Sep 21 '19 at 00:14

James R. Perkins

16,800
44
60

Wow, that was a simple solution. For some reason I thought filter-spec was only allowed under handlers, not loggers. I thought I tried the filter with the root logger and it didn't work (is that expected?) but it works fine with my custom loggers. Thanks. – Drew Nutter Sep 23 '19 at 15:39
1

Originally, and still in WildFly, loggers don't inherit filters. So if you need to filter a log with name `org.jboss.example.SomeClass` you'd need to create that specific logger and add the filter to that logger. In the log manager filters can now be inherited, however that behavior is not yet exposed in WildFly. – James R. Perkins Sep 24 '19 at 16:25

JBoss/Wildfly - Safe and efficient log filtering - Multiple handlers logging to the same file

1 Answers1