Logstash/Gelf not stored all MDCs

Question

I'm using Logstash/Gelf as a log-tools in my application. I follow this sample for setting up my logger as JSON file. here is my wildfly config (standalone.xml):

<profile>
    <subsystem xmlns="urn:jboss:domain:logging:3.0">
        <periodic-rotating-file-handler name="JsonLog" autoflush="true">
            <level name="ALL"/>
            <formatter>
                <named-formatter name="JsonFormatter"/>
            </formatter>
            <file relative-to="jboss.server.log.dir" path="server.json"/>
            <suffix value=".yyyy-MM-dd"/>
            <append value="true"/>
        </periodic-rotating-file-handler>
        ...
        <logger category="org.somepackgs.SoapLogHandler" use-parent-handlers="true">
            <level name="INFO"/>
            <handlers>
                <handler name="JsonLog"/>
            </handlers>
        </logger>
        ...
        <formatter name="JsonFormatter">
            <custom-formatter module="biz.paluch.logging" class="biz.paluch.logging.gelf.wildfly.WildFlyJsonFormatter">
                <properties>
                    <property name="version" value="1.0"/>
                    <property name="facility" value="java-test"/>
                    <property name="fields" value="Time,Severity"/>
                    <property name="extractStackTrace" value="true"/>
                    <property name="filterStackTrace" value="true"/>
                    <property name="includeLogMessageParameters" value="true"/>
                    <property name="includeLocation" value="true"/>
                    <property name="mdcProfiling" value="true"/>
                    <property name="timestampPattern" value="yyyy-MM-dd HH:mm:ss,SSS"/>
                    <property name="mdcFields" value="requestId"/>
                    <property name="dynamicMdcFields" value="mdc.*,(mdc|MDC)fields"/>
                    <property name="includeFullMdc" value="true"/>
                </properties>
            </custom-formatter>
        </formatter>
    </subsystem>
</profile>

In SoapLogHandler I put some fields in MDC, my application has aGeneralInterceptor for checking some business logic and I want log some extra fields (like result, duration time) after process request:

@AroundInvoke
public Object run(InvocationContext ictx) throws Exception {
    MDC.put("requestId", UUID.randomUUID());
    try{
        //other codes & business checks...
        obj = ictx.proceed();
     } catch (Exception ex) {

    } finally {
        logAfterInvokation(ictx);
    }
}

in logAfterInvokation method, I put some extra fields (like result, duration time) in MDC; But when line obj = ictx.proceed(); runs, Logstash automatically store some fields in server.json file and my extra fields never adds to this file, here is one node of my server.json file:

{
    "short_message": "short message text...",
    "full_message": "full message text...",
    "timestamp": "1558244976.376",
    "level": "6",
    "facility": "java-test",
    "logType": "entry",
    "LoggerName": "org.somepackgs.SoapLogHandler",
    "fieldName1": "fieldValue1",
    "destination": "destination1",
    "Time": "2019-05-19 10:19:36,376",
    "Severity": "INFO",
    "source": "my-App",
    "Thread": "default task-1",
    "SourceMethodName": "logMessage",
    "SourceSimpleClassName": "SoapLogHandler",
    "requestId": "d4d8482f-e9ae-4f31-acce-1629515a2ffd",
    "SourceClassName": "org.somepackgs.SoapLogHandler"
}

So why Logstash not store all of MDC contents? or How I can store all of my MDC contents to json file? Any help would be appreciated!

Updated: I think my JsonLog (in standalone.xml) placed in wrong location, Becuase It logs org.somepackgs.SoapLogHandler and the main application's flow occurs in interceptor, but when I place JsonLog for org.somepackgs.GeneralInterceptor, the SOAP-XML not logged anymore.

can you add the field names which you add in `mdc` or the code as to how you add those? — swayamraina, May 19 '19 at 07:35
I put in MDC in regular way: `MDC.put("methodName", ictx.getMethod().getName());` — Morteza Asadi, May 19 '19 at 07:39
`org.somepackgs.SoapLogHandler` This is the log implementation class right? It should not have to do anything with where you add things to MDC. Can you test my updating the regex. — swayamraina, May 19 '19 at 07:51

score 0 · Answer 1 · answered May 19 '19 at 07:52

0

<property name="dynamicMdcFields" value="mdc.*,(mdc|MDC)fields"/>

The dynamicMdcFields evaluates every value you put through a regex that you provide in its value.
Probably, you need to update your regex or update field names that you put in MDC

answered May 19 '19 at 07:52

swayamraina

2,958
26
28

yes, you either need to change the value to `[a-zA-Z0-9]*` or update `MDC.put("methodName", ictx.getMethod().getName());` to `MDC.put("mdc.methodName", ictx.getMethod().getName());` – swayamraina May 19 '19 at 08:02
I changed that line as `` but in json file nothings changed, it won't work. In any case, thank you in advance! – Morteza Asadi May 19 '19 at 08:10
what is the output of `ictx.getMethod().getName()` ? – swayamraina May 19 '19 at 08:12
`ictx` is `InvocationContext` default argument of Interceptor, `ictx.getMethod().getName()` returns method's name (in `String`) that Interceptor invoke during call this method. – Morteza Asadi May 19 '19 at 08:15

score 0 · Accepted Answer · answered Jun 18 '19 at 04:57

I placed my JsonLog (in standalone.xml) in wrong location, Becuase all of MDCs is not initialized when org.somepackgs.SoapLogHandler runs, So I need create a class with name FinalLogger and call one method of this class for logging after obj = ictx.proceed(); line, something like below:

FinalLogger.logInfo("some text.");

and FinalLogger is like below:

package org.somepackgs;
import org.apache.log4j.Logger;

public class FinalLogger {
    private static Logger logger = Logger.getLogger(FinalLogger.class);

    public static void logInfo(String message) {
        logger.info(message);
    }
}

and in standalone.xml put jsonLog on FinalLogger:

<logger category="org.somepackgs.FinalLogger" use-parent-handlers="true">
    <level name="INFO"/>
    <handlers>
        <handler name="JsonLog"/>
    </handlers>
</logger>

Because the FinalLogger call at end of every rquest, So in time of running this all MDCs has been initialized and at this time stored in json log file.

Logstash/Gelf not stored all MDCs

2 Answers2