1

I have got an issue with my application, it logs request along with its query param which may contain sensitive data in access log. application is configured with logback.xml & embedded jetty.

jetty server is customized with below accessLogCustomer

public JettyServerCustomizer accessLogCustomizer() {
  return server -> {
    Slf4jRequestLog requestLog = new Slf4jRequestLog();
    requestLog.setExtended(true);
    requestLog.setLogLatency(true);
    requestLog.setPreferProxiedForAddress(true);
    requestLog.setLogTimeZone(userTimezone == null ? ZoneId.systemDefault().getId() : userTimezone);
    requestLog.setLogDateFormat("Y-MM-dd HH:mm:ss, SSS Z");

    RequestLogHandler requestLogHandler = new RequestLogHandler();
    requestLogHandler.setRequestLog(requestLog);
    requestLogHandler.setHandler(server.getHandler());
    server.setHandler(requestLogHandler);
  };
}

logback.xml

    <appender name="access" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <File>${logs.dir}/abc-access.log</File>
        <encoder class="ch.qos.logback.core.encoder.LayoutWrappingEncoder">
            <layout class="ch.qos.logback.classic.PatternLayout">
                <Pattern>%m %n</Pattern>
            </layout>
            <charset>UTF-8</charset>
        </encoder>
        <rollingPolicy class="ch.qos.logback.core.rolling.TimeBasedRollingPolicy">
            <FileNamePattern>${logs.dir}/abc-access.%d.log.gz</FileNamePattern>
        </rollingPolicy>
    </appender>

    <logger name="org.eclipse.jetty.server.RequestLog" additivity="false">
        <appender-ref ref="access"/>
    </logger>

request logged in access log

192.168.0.100 - - [2021-05-20 15:48:15,093 +0530] "POST /myAPI/v2/customer/message?myID=123&messageText=hello HTTP/1.0" 200 0 "-" "PostmanRuntime/7.26.8" 475

I am trying to avoid messageText from access log, but not getting any solution.

piyush vijayvargiya
  • 87
  • 1
  • 11

1 Answers1

0

Use the CustomRequestLog and Slf4jRequestLogWriter instead.

You'll want the special format option %U which emits the URL path, without the query string (which is available as %q btw)

Your resulting configuration would look like this ...

Slf4jRequestLogWriter slfjRequestLogWriter = new Slf4jRequestLogWriter();
String format = "%{client}a - %u %t %m \"%U\" %s %O \"%{Referer}i\" \"%{User-Agent}i\"";
CustomRequestLog customRequestLog = new CustomRequestLog(slfjRequestLogWriter, format);
server.setRequestLog(customRequestLog);

Play with the format line, read the Javadoc on CustomRequestLog to know what you can do.

Some notes:

  • The example format is not strictly following the Extended NCSA format (as it's missing the HTTP version portion, and the HTTP method is outside of the quoted section, but that is usually not a problem for many users)
  • Slf4jRequestLogWriter is only concerned with taking the formatted log line and sending it to the slf4j-api, it does nothing else.
  • RequestLogHandler is deprecated and not a recommended usage anymore (as it does not log bad requests and context-less requests), use the Server.setRequestLog(RequestLog) instead.

Jetty will use the CustomRequestLog's Pattern to produce a String, this String is forwarded to the Slf4jRequestLogWriter as a slf4j logging event message, which is then logged per your existing slf4j + logback configuration.

Joakim Erdfelt
  • 46,896
  • 7
  • 86
  • 136
  • thanks for the response @joakim Erdfelt, but here classsic.PatternLayout class is being used, which doesn't support %u and other formats. http://logback.qos.ch/xref/ch/qos/logback/classic/PatternLayout.html – piyush vijayvargiya May 31 '21 at 12:34
  • You are getting yourself confused, the pattern I wrote about is not the logback one, but rather the CustomRequestLog one, the one in Jetty itself. The logback one is only concerned with taking what CustomRequestLog + Slf4jRequestLogWriter is producing as the message and wrapping it in traditional logging stuff (like timestamps, thread ids, logger names, logger levels, etc.) The pattern I'm referring to in my answer is 100% within Jetty itself and slf4j or logback is never aware of it. – Joakim Erdfelt Jun 01 '21 at 12:17
  • **JoakimErdfelt** thanks for putting my focus on the working of CustomRequestLog. now, I have come across a situation where I don't want to skip all the query param, instead I want to remove query param which contains user data for example : `192.168.0.100 - - [2021-05-20 15:48:15,093 +0530] "POST /myAPI/v2/customer/message?myID=123&messageText=hello HTTP/1.0" 200 0 "-" "PostmanRuntime/7.26.8" 475` I want to keep myID but **not messageText** ? – piyush vijayvargiya Jun 09 '21 at 15:34
  • **JoakimErdfel** also, is there any way if we can mask data in messageText ? – piyush vijayvargiya Jun 10 '21 at 09:33
  • Use logback's replace features to filter/mask it out - https://stackoverflow.com/questions/25052590/logback-replacing-10-digit-number-by-and-last-two-digit – Joakim Erdfelt Jun 10 '21 at 20:01
  • This solution masks all the number matching the regex condition we have defined in logback.xml as : `%replace(%msg){'\b\d{3}\b','***'} %n`. but my requirement is to keep only query param with name `myID` along with its value. and other query param either can be masked or can be removed. – piyush vijayvargiya Jun 11 '21 at 05:19
  • logback replace feature is a generic solution (using regex) that you can use to filter out/replace any content from the output of logback. It is 100% possible to create the right regex for your situation. – Joakim Erdfelt Jun 11 '21 at 11:35