0

I have this on sender side (log4j.properties file for jboss):

log4j.appender.LOGSTASH=org.apache.log4j.net.SocketAppender
log4j.appender.LOGSTASH.layout=org.apache.log4j.PatternLayout
log4j.appender.LOGSTASH.layout.ConversionPattern=%-5p %t %d{ISO8601} %l - %m%n
log4j.appender.LOGSTASH.RemoteHost=my.server.com
log4j.appender.LOGSTASH.ReconnectionDelay=60000
log4j.appender.LOGSTASH.Threshold=DEBUG

But on receiving side I always receive some default message, ConversionPattern doesnt work, even I remove everything and put just a text there. I have four jboss instances, so it is critical to me to define on every sender side some specific text, e.g. log4j.appender.LOGSTASH.layout.ConversionPattern=server1 %-5p %t %d{ISO8601} %l - %m%n

Serge
  • 341
  • 1
  • 4
  • 7

3 Answers3

1

SocketAppender doesn't allow a PatternLayout see: How to use Pattern layout with SocketAppender

You'll probably need to use the SyslogAppender or the TelnetAppender to do what you want.

Community
  • 1
  • 1
Alcanzar
  • 16,985
  • 6
  • 42
  • 59
  • Thanks. Yes, I know that I can deserialize logs on receiver side, but it is quite time consuming :( but seems I dont have any other option. I will use shipper.conf for Logstash (on receiving side) for these purposes. – Serge Jun 17 '14 at 05:15
  • You may run into issues with the SyslogAppender as most syslog servers will truncate messages longer than 1024 chars by default. That would include stack-traces that you'd obviously want to capture. – stuart-warren Jun 17 '14 at 19:28
  • You can use the SyslogAppender to log directly to logstash's syslog input without worrying about that I think (I've never done that, but it should work) – Alcanzar Jun 17 '14 at 19:36
0

Or write your own appender, It's actually pretty easy. Just extend org.apache.log4j.AppenderSkeleton.

Here is my attempt at a ZeroMQ (http://zeromq.org/intro:read-the-manual) appender and JSON (needs minimal processing) layout that I use for Logstash.

https://github.com/stuart-warren/logit

Produces output similar to:

{"message":"Hello World!","test":"other","@timestamp":"2013-10-06T21:22:09.868Z","level":"DEBUG","mdc":{"field_name_here":"detail goes here"},"thread":"main","logger":"com.stuartwarren.test_logit.log4j1.LogIt","@version":"1"}

Note: far from perfect, but not a bad first shot for an infrastructure guy.

stuart-warren
  • 605
  • 5
  • 14
  • Sorry, but I cannot use your proposal, because it is enterprise environment. – Serge Jun 17 '14 at 05:13
  • Which proposal exactly? Writing your own appender/layout or using the hacky one I linked to? Can you edit your question to state what your limits are, then we can give a more suitable answer. – stuart-warren Jun 17 '14 at 18:23
0

The Logstash log4j plugin https://github.com/elasticsearch/logstash/blob/v1.4.1/lib/logstash/inputs/log4j.rb that I assume you are using, appears to extract the hostname from the sending server to the host field. Is that not sufficient?

It should also extract objects from the MDC and NDC log4j structures, if it is an internal application your developers could set something in there, or allow you to set a value in the startup params. https://logging.apache.org/log4j/1.2/apidocs/org/apache/log4j/MDC.html

With my com.stuartwarren.logit.log4j1.Layout code you can just add your custom fields in the config file, but the SocketAppender doesn't appear to give you that ability as you can't set/change the layout.

stuart-warren
  • 605
  • 5
  • 14