How to modify log4j JsonLayout field names

Question

I have configured log4j 2 with this config file to write my MapMessage to console:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="ERROR">
    <Properties>
        <Property name="disableThreadContext">true</Property>
        <Property name="disableThreadContextStack">true</Property>
        <Property name="disableThreadContextMap">true</Property>
        <Property name="log4j2.disable.jmx">true</Property>
    </Properties>
    <Appenders>
        <Console name="CONSOLE" target="SYSTEM_OUT">
            <JsonLayout locationInfo="true" complete="false" />
        </Console>
    </Appenders>
    <Loggers>
        <logger name="ir.cvas.logger" level="info" />
        <Root level="warn">
            <AppenderRef ref="CONSOLE"/>
        </Root>
    </Loggers>
</Configuration>

The output of this configuration is like:

{
  "timeMillis" : 1404902036494,
  "thread" : "main",
  "level" : "ERROR",
  "loggerName" : "ir.cvas.log4j.json.Main",
  "message" : "description=\"I'm so fucked...\" id=\"12312312312312321\"",
  "endOfBatch" : false,
  "loggerFqcn" : "org.apache.logging.log4j.spi.AbstractLogger",
  "source" : {
    "class" : "ir.cvas.log4j.json.Main",
    "method" : "main",
    "file" : "Main.java",
    "line" : 24
  }
}

1. I need to remove some fields from output such as endOfBatch and loggerFqcn. Is it possible?
2. I don't really like how MapMessage is converted. I would want something like Json object instead of single string, Something like thisthis: "message": { "description"="I'm so fucked...", "id"="12312312312312321" } or at least flatten MapMessage fields in to log message.
3. I want to convert timestamp field to unix time with milliseconds removed.

Can anyone help?

Answer 1

user JsonTemplateLayout:

Pom.xml:

<dependency>
    <groupId>org.apache.logging.log4j</groupId>
    <artifactId>log4j-layout-template-json</artifactId>
    <version>2.14.1</version>
</dependency>

log4j2.xml:

<?xml version="1.0" encoding="UTF-8"?>
<Configuration status="INFO">
    <Appenders>
        <Console name="console" target="SYSTEM_OUT">
            <JsonTemplateLayout eventTemplateUri="classpath:JsonTemplateLayout.json" />
        </Console>
    </Appenders>
    <Loggers>
        <Logger name="com.logging.a.b" level="error" additivity="false">
            <AppenderRef ref="console" />
        </Logger>
    </Loggers>
</Configuration>

src/main/resources/JsonTemplateLayout.json:

{
    "version": "1.1",
    "hostName": "${hostName}",
    "field1": "${env:JAVA_HOME}",
    "field2": "${sys:os.name}",
    "time": {
        "$resolver": "timestamp",
        "pattern": {
            "format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
            "timeZone": "UTC"
        }
    },
    "level": {
        "$resolver": "level",
        "field": "name"
    },
    "loggerName": {
        "$resolver": "logger",
        "field": "name"
    },
    "message": {
        "$resolver": "message",
        "stringified": true
    },
    "source": {
        "class": {
            "$resolver": "source",
            "field": "className"
        },
        "method": {
            "$resolver": "source",
            "field": "methodName"
        },
        "file": {
            "$resolver": "source",
            "field": "fileName"
        },
        "line": {
            "$resolver": "source",
            "field": "lineNumber"
        }
    },
}

You can add and remove fields, or change name of fields by changing the above template file. See more examples here: json template file

Answer 2

You can customize JacksonFactory class to include only field names that you want in log entry. Default implementation in JacksonFactory is excluding only two fields from log entry which are location and properties (that too based on parameters properties and source). But if you want to exclude more fields you could add more field names to exclude in set of properties to exclude, Here is the example

ObjectWriter newWriter(final boolean locationInfo, final boolean properties, final boolean compact) {
        final SimpleFilterProvider filters = new SimpleFilterProvider();
        final Set<String> except = new HashSet<>(5);
        if (!locationInfo) {
            except.add(this.getPropertNameForSource());
        }
        if (!properties) {
            except.add(this.getPropertNameForContextMap());
    }

    except.add(this.getPropertNameForNanoTime());

    except.add("loggerFqcn");
    except.add("endOfBatch");
    filters.addFilter(Log4jLogEvent.class.getName(), SimpleBeanPropertyFilter.serializeAllExcept(except));

    final ObjectWriter writer = this.newObjectMapper().writer(compact ? this.newCompactPrinter() : this.newPrettyPrinter());

    return writer.with(filters);
}

Answer 3

I'm basing my response on this manual reference.

1) Apparently the only parameters the JSONLayout takes are charset, compact, complete, properties, and locationInfo. Unfortunately based on my limited research, removing endOfBatch and loggerFqcn doesn't appear possible. (This is odd, however, since the example output in the referenced manual does not include these fields.)

2) This is almost certainly not possible. The concept of a log entry in log4j involves a string message, possibly a throwable, a log level, and a few other static things. Custom fields aren't possible without writing a custom class. This SO question describes doing that, though for the purpose of that question, MDC was used in the end. MDC wouldn't serve your purposes since you want to have custom fields that are different in each entry. I'm not sure what you mean by "flatten."

3) Unfortunately this doesn't look like it's available either. PatternLayout is much more customizable; AFAIK you will have to customize JSONLayout yourself.

Answer 4

It is possible to do this with a Log4j extension. See https://github.com/metamx/sumologic-log4j2-appender/blob/master/src/main/java/com/sumologic/log4j/core/SumoJsonLayout.java#L117-L237 for an example. Basically you can extend LogEvent with whatever Jackson annotations you want to customize how things are printed out, but then you need some kind of layout in order to coerce the events into your custom-field-named event object.

Answer 5

You can use the solution in the below link to customize these attributes.

https://stackoverflow.com/a/76303815/11092008