2

Inspired by this Java class approach I am trying an XML based approach for sending logs to ELK directly through logback-spring.xml. This fails and the logs don't reach ELK. What could be the reason?Any help would be deeply appreciated.

@Configuration approach

    private void addLogstashAppender(LoggerContext context) {
    LogstashTcpSocketAppender logstashAppender = new LogstashTcpSocketAppender();
    logstashAppender.setName(LOGSTASH_APPENDER_NAME);
    logstashAppender.setContext(context);
    String customFields = "{\"app_name\":\"" + appName + "\",\"app_port\":\"" + serverPort + "\"}";
    LogstashEncoder logstashEncoder=new LogstashEncoder();
    // Set the Logstash appender config from JHipster properties
    logstashEncoder.setCustomFields(customFields);
    // Set the Logstash appender config from JHipster properties
    logstashAppender.addDestinations(new InetSocketAddress(jHipsterProperties.getLogging().getLogstash().getHost(),jHipsterProperties.getLogging().getLogstash().getPort()));

    ShortenedThrowableConverter throwableConverter = new ShortenedThrowableConverter();
    throwableConverter.setRootCauseFirst(true);
    logstashEncoder.setThrowableConverter(throwableConverter);
    logstashEncoder.setCustomFields(customFields);

    logstashAppender.setEncoder(logstashEncoder);
    logstashAppender.start();

    // Wrap the appender in an Async appender for performance
    AsyncAppender asyncLogstashAppender = new AsyncAppender();
    asyncLogstashAppender.setContext(context);
    asyncLogstashAppender.setName(ASYNC_LOGSTASH_APPENDER_NAME);
    asyncLogstashAppender.setQueueSize(jHipsterProperties.getLogging().getLogstash().getQueueSize());
    asyncLogstashAppender.addAppender(logstashAppender);
    asyncLogstashAppender.start();
    context.getLogger("ROOT").addAppender(asyncLogstashAppender);
}

logback-spring.xml approach (doesn't send logs to ELK)

<appender name="LOGSTASH"
          class="net.logstash.logback.appender.LogstashTcpSocketAppender">
    <destination>${LOGSTASH_HOST}:${LOGSTASH_PORT}</destination>
    <encoder charset="UTF-8"
             class="net.logstash.logback.encoder.LogstashEncoder">
    </encoder>
</appender>   
<root level="info">
    <appender-ref ref="FILE-ROLLING"/>
    <appender-ref ref="LOGSTASH"/>
</root>
<shutdownHook class="ch.qos.logback.core.hook.DelayingShutdownHook"/>

I ran the application in debug mode after removing the AsyncAppender and the logs are as below. Nothing untoward noticed.

22:33:10,313 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [net.logstash.logback.appender.LogstashTcpSocketAppender]
22:33:10,379 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [LOGSTASH]
22:33:11,090 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [com.sampleapp] to INFO
22:33:11,093 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating INFO level on Logger[com.sampleapp] onto the JUL framework
22:33:11,101 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [com.sampleapp] to false
22:33:11,102 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE-ROLLING] to Logger[com.sampleapp]
22:33:11,105 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.hibernate] to INFO
22:33:11,105 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating INFO level on Logger[org.hibernate] onto the JUL framework
22:33:11,106 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.hibernate] to false
22:33:11,106 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE-ROLLING] to Logger[org.hibernate]
22:33:11,106 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.hibernate.SQL] to DEBUG
22:33:11,107 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating DEBUG level on Logger[org.hibernate.SQL] onto the JUL framework
22:33:11,108 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.hibernate.SQL] to false
22:33:11,109 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE-ROLLING] to Logger[org.hibernate.SQL]
22:33:11,111 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.hibernate.type.descriptor.sql] to DEBUG
22:33:11,112 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating DEBUG level on Logger[org.hibernate.type.descriptor.sql] onto the JUL framework
22:33:11,112 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.hibernate.type.descriptor.sql] to false
22:33:11,113 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE-ROLLING] to Logger[org.hibernate.type.descriptor.sql]
22:33:11,114 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting level of logger [org.springframework] to INFO
22:33:11,114 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating INFO level on Logger[org.springframework] onto the JUL framework
22:33:11,114 |-INFO in ch.qos.logback.classic.joran.action.LoggerAction - Setting additivity of logger [org.springframework] to false
22:33:11,114 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE-ROLLING] to Logger[org.springframework]
22:33:11,115 |-INFO in ch.qos.logback.classic.joran.action.RootLoggerAction - Setting level of ROOT logger to INFO
22:33:11,116 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating INFO level on Logger[ROOT] onto the JUL framework
22:33:11,116 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [FILE-ROLLING] to Logger[ROOT]
22:33:11,116 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [LOGSTASH] to Logger[ROOT]
22:33:11,116 |-INFO in ch.qos.logback.core.joran.action.ShutdownHookAction - About to instantiate shutdown hook of type [ch.qos.logback.core.hook.DelayingShutdownHook]
22:33:11,122 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
22:33:11,123 |-INFO in org.springframework.boot.logging.logback.SpringBootJoranConfigurator@448c8166 - Registering current configuration as safe fallback point
22:33:11,179 |-INFO in net.logstash.logback.appender.LogstashTcpSocketAppender[LOGSTASH] - Log destination localhost:4560: connection established.
22:33:11,192 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@565f390 - Propagating DEBUG level on Logger[com.sampleapp] onto the JUL framework
raikumardipak
  • 1,461
  • 2
  • 29
  • 49
  • 2
    Couple comments. 1) you don't need to wrap the `LogstashTcpSocketAppender` in an `AsyncAppender` because `LogstashTcpSocketAppender` is _already_ asynchronous 2) if you turn on logback debugging ``, do you see any errors? – Phil Clay May 13 '20 at 16:25
  • @PhilClay Nothing untoward noticed Phil. I have attached the logs – raikumardipak May 13 '20 at 17:16
  • i tried that after removing async_appender. Any clue would be great @PhilClay. Stuck badly with the logback-spring.xml approach. It works in Logging Configuration.java approach. Not sure if there is some other issue – raikumardipak May 13 '20 at 17:27
  • Does this affect the loading of logback-spring.xml – raikumardipak May 13 '20 at 17:40
  • What is running at localhost:4560? For debugging, perhaps try running a simple netcat server (`nc -lk 4560`) instead, to see if it sees any logs. – Phil Clay May 25 '20 at 20:38

0 Answers0