11

We currently generically log all XML documents coming in and going out of our system, and some of them contain passwords in the clear. We would like to be able to configure the logback logger/appender that is doing this to do some pattern matching or similar and if it detects a password is present to replace it (with asterisks most likely). Note we don't want to filter out the log entry, we want to mask a portion of it. I would appreciate advice on how this would be done with logback. Thanks.

SingleShot
  • 18,821
  • 13
  • 71
  • 101

2 Answers2

23

The logback version 0.9.27 introduced replacement capability. Replacements support regular expressions. For example, if the logged message was "userid=alice, pswd='my secret'", and the output pattern was 

  "%d [%t] $logger - %msg%n",

you just modify the pattern to 

 "%d [%t] $logger - %replace(%msg){"pswd='.*'", "pswd='xxx'"}%n"

Note that the above makes use of option quoting.

The previous log message would be output as "userid=alice, pswd='xxx'"

For blazing performance, you could also mark the log statement as CONFIDENTIAL and instruct %replace to perform replacement only for log statements marked as CONFIDENTIAL. Example,

 Marker confidential = MarkerFactory.getMarker("CONFIDENTIAL");
 logger.info(confidential, "userid={}, password='{}'", userid, password);

Unfortunately, the current version of logback does not yet support conditional replacements (based on markers or otherwise). However, you could easily write your own replacement code by extending ReplacingCompositeConverter. Shout on the logback-user mailing list if you need further assistance.

Ceki
  • 26,753
  • 7
  • 62
  • 71
  • 1
    Great! I knew something like this had to be there in your excellent product. Thanks for your contributions to the Java world! – SingleShot Jan 07 '11 at 20:00
  • @Ceki Do you know if `%replace(%msg){...}` works combined with evaluator expressions? I.e. if I define something like `message.contains("pw=") || message.contains("password=")` can I apply the replacement rules only on log-lines that meet the CONFIDENTIAL expression evaluator? I.e. something like `%replace(%msg{CONFIDENTIAL}){'(pw|password)=.*?(%amp;|$)', '$1=XXX$2'}` the log statements are masked properly, though it feels like the replacement is attempted on each log line based on the execution time – Roman Vottner Jun 22 '17 at 12:23
  • `%amp;` should of course be `&` typo on my part but can't edit the comment anymore – Roman Vottner Jun 22 '17 at 12:30
  • The one thing that comes to my mind is how much performance impact this replacement might have. Anyone have any idea? – Raja Anbazhagan Jan 25 '22 at 11:34
0

I believe Masking is an aspect of your business, not the aspect of any technology or logging system. There are situations where the passwords, national identities etc should be masked while storing them in the DB as well due to legal reasons. You should be able to mask the xml before giving it to the logger.

One way to do it is to run the XML through XSLT that does that making and then give it to logger for logging.

If you doesn't want to do this then LogBack has Filters support that is one of the option (not the right one though).

But understand that any generic out of the box solution you are trying to find at the logging infrastructure level is going to be suboptimal as every log message is going to be checked for masking.

Aravind Yarram
  • 78,777
  • 46
  • 231
  • 327
  • Thanks. I agree, though in this case I would prefer to use Logback if it has this capability. Pretend its not a password or XML and I just want to replace some text being logged. That's what I want to know how to do. – SingleShot Jan 05 '11 at 21:55
  • Filters are for discarding unwanted log events. I don't want to discard log events but want to filter/replace/mask log event messages if they meet certain criteria. – SingleShot Jan 05 '11 at 22:17
  • I Agree but just wanted to share that if you didn't find any valid options then this can be one of the valid options – Aravind Yarram Jan 05 '11 at 22:28
  • Thanks for your help. Its only suboptimal at a particular logger (or appender) instance level (the logger/appender that logs XML messages), but yes. – SingleShot Jan 05 '11 at 23:00