InternetItem's ElasticsearchAppender properties are ignored by Logback 1.3.5

Question

Migrating from ElasticsearchAppender 1.1 to 1.3.5 and using the example configuration suggested in https://github.com/internetitem/logback-elasticsearch-appender:

<appender name="ELASTIC" 
  class="com.internetitem.logback.elasticsearch.ElasticsearchAppender">
  <url>http://yourserver/_bulk</url>
  <index>logs-%date{yyyy-MM-dd}</index>
  <type>tester</type>
  blah ... blah ...
    <properties>
        <property>
            <name>host</name>
            <value>${HOSTNAME}</value>
            <allowEmpty>false</allowEmpty>
        </property>
        <property>
            <name>severity</name>
            <value>%level</value>
        </property>
        <property>
            <name>thread</name>
            <value>%thread</value>
        </property>
        <property>
            <name>stacktrace</name>
            <value>%ex</value>
        </property>
        <property>
            <name>logger</name>
            <value>%logger</value>
        </property>
    </properties>
 blah ... blah ...

Step debugging thro the Logback configuration initialization stage, I found that the properties block is completely ignored by Logback model joran model reading stage.

The model handler is treating the element "<property " as a Logback reserved model, rather than going forth to the ElasticsearchAppender properties bean to search for the appropriate adder method, Logback complains each property structure in the ElasticsearchAppender xml block is not recognized as valid.

Is this a defect, or does InternetItem have a new ElasticsearchAppender configuration compatible with Logback 1.3 ++?

Jens Rieks · Answer 1 · 2023-03-20T12:29:25.347

2

The internetitem version of logback-elasticsearch-appender seems to be unmaintained, so we made a fork here: https://github.com/agido-malter/logback-elasticsearch-appender/

Your problem has been solved there, you can now use <esProperty> instead of <property>.

The fix is included in the newest version here: https://central.sonatype.com/artifact/com.agido/logback-elasticsearch-appender/3.0.6

edited Mar 20 '23 at 12:29

answered Mar 20 '23 at 07:33

Jens Rieks

21
2

score 0 · Answer 2 · answered Mar 19 '23 at 08:10

By step debugging thro the Logback initialization phase, I discovered that the issue arose because I had upgraded from Logback 1.1.x to 1.3.x and 1.4.x.

Logback 1.3 onwards reserves the use of the element name "property" for its own property settings.

Therefore I had to create a wrapper class to re-express the element name "property" as something else. I decided to use "attribute".

package yourcompany.blahblah;
import com.internetitem.logback.elasticsearch.ElasticsearchAppender;
import com.internetitem.logback.elasticsearch.config.*;

public class LogbackCompatibilityWrapper4ElasticsearchAppender
  extends ElasticsearchAppender {
  public void setAttributes(Attributes attributes) {
    super.setProperties(attributes);
  }

  static public class Attribute extends Property {
    /* No-args constructor must be present 
     *  because Logback instatiates it to use the setters.
     */
    public ElasticsearchProperties () {}

    public ElasticsearchProperties (
      String name, String value, boolean allowEmpty) {
        super(name, value, allowEmpty);
    }
  }

  static public class Attributes extends ElasticsearchProperties {
    public void addAttribute(Attribute attribute) {
      super(attribute);
    }
  }
}

So that the appender config in the ElasticserchAppender github page should be changed to the manner of

<appender name="ELASTIC" class="yourcompany.blahblah.LogbackCompatibilityWrapper4ElasticsearchAppender">
    <url>http://yourserver/_bulk</url>
    <index>logs-%date{yyyy-MM-dd}</index>
    <type>tester</type>
    <loggerName>es-logger</loggerName> <!-- optional -->
    <errorLoggerName>es-error-logger</errorLoggerName> <!-- optional -->
    <connectTimeout>30000</connectTimeout> <!-- optional (in ms, default 30000) -->
    <errorsToStderr>false</errorsToStderr> <!-- optional (default false) -->
    <includeCallerData>false</includeCallerData> <!-- optional (default false) -->
    <logsToStderr>false</logsToStderr> <!-- optional (default false) -->
    <maxQueueSize>104857600</maxQueueSize> <!-- optional (default 104857600) -->
    <maxRetries>3</maxRetries> <!-- optional (default 3) -->
    <readTimeout>30000</readTimeout> <!-- optional (in ms, default 30000) -->
    <sleepTime>250</sleepTime> <!-- optional (in ms, default 250) -->
    <rawJsonMessage>false</rawJsonMessage> <!-- optional (default false) -->
    <includeMdc>false</includeMdc> <!-- optional (default false) -->
    <maxMessageSize>100</maxMessageSize> <!-- optional (default -1 -->
    <authentication class="com.internetitem.logback.elasticsearch.config.BasicAuthentication" /> <!-- optional -->
    <attributes>
        <attribute>
            <name>host</name>
            <value>${HOSTNAME}</value>
            <allowEmpty>false</allowEmpty>
        </attribute>
        <attribute>
            <name>severity</name>
            <value>%level</value>
        </attribute>
        <attribute>
            <name>thread</name>
            <value>%thread</value>
        </attribute>
        <attribute>
            <name>stacktrace</name>
            <value>%ex</value>
        </attribute>
        <attribute>
            <name>logger</name>
            <value>%logger</value>
        </attribute>
    </attributes>
    <headers>
        <header>
            <name>Content-Type</name>
            <value>application/json</value>
        </header>
    </headers>
</appender>

<root level="info">
    <appender-ref ref="FILELOGGER" />
    <appender-ref ref="ELASTIC" />
</root>

<logger name="es-error-logger" level="INFO" additivity="false">
    <appender-ref ref="FILELOGGER" />
</logger>

<logger name="es-logger" level="INFO" additivity="false">
    <appender name="ES_FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
        <!-- ... -->
        <encoder>
            <pattern>%msg</pattern> <!-- This pattern is important, otherwise it won't be the raw Elasticsearch format anyomre -->
        </encoder>
    </appender>
</logger>

After which the properties I configured started to appear in the message sent to logstash.

InternetItem's ElasticsearchAppender properties are ignored by Logback 1.3.5

2 Answers2