Using Pulsar as appender to Log4j2: How to stop logging acknowledgements?

Question

I have a Producer that logs messages using Apache Pulsar as an appender to Log4j2.

The Consumer is listening on the same Pulsar topic and writing the received messages to a log file. The Consumer is able to receive the messages properly, but the problem is it's also logging other stuff with it too:

[localhost/127.0.0.1:6650] Received cmd LOOKUP_RESPONSE
Received Broker lookup response: Connect
[coinflex_logs][coinflex_logs_sub] Subscribing to topic on cnx [id: 0xb78fd850, L:/127.0.0.1:53782 - R:localhost/127.0.0.1:6650]
[localhost/127.0.0.1:6650] Received cmd SUCCESS
[id: 0xb78fd850, L:/127.0.0.1:53782 - R:localhost/127.0.0.1:6650] Received success response from server: 2
[coinflex_logs][coinflex_logs_sub] Subscribed to topic on localhost/127.0.0.1:6650 -- consumer: 0
[coinflex_logs] [coinflex_logs_sub] Adding 1000 additional permits
[localhost/127.0.0.1:6650] Received cmd MESSAGE
[id: 0xb78fd850, L:/127.0.0.1:53782 - R:localhost/127.0.0.1:6650] Received a message from the server: org.apache.pulsar.common.api.proto.PulsarApi$CommandMessage@2dab32d1
[coinflex_logs][coinflex_logs_sub] Received message: 3828/25
SSE4.2 CRC32C provider initialized
[coinflex_logs_sub] [aa9ca] processing message num - 0 in batch
[coinflex_logs_sub] [aa9ca] enqueued messages in batch. queue size - 0, available queue size - 2147483647
15:23:59.459 [main] DEBUG com.pulsar_logging_producer.Producer -==========TEST MESSAGE=============

[coinflex_logs_sub] [aa9ca] can ack message to broker 3828:25:-1:0, acktype Individual, cardinality 0, length 1
[coinflex_logs_sub] [aa9ca] acknowledging message - 3828:25:-1:0, acktype Individual
[ConsumerBase{subscription='coinflex_logs_sub', consumerName='aa9ca', topic='coinflex_logs'}] Flushing pending acks to broker: last-cumulative-ack: -1:-1:-1 -- individual-acks: []
[localhost/127.0.0.1:6650] Received cmd PING
[[id: 0xb78fd850, L:/127.0.0.1:53782 - R:localhost/127.0.0.1:6650]] Replying back to ping message
[[id: 0xb78fd850, L:/127.0.0.1:53782 - R:localhost/127.0.0.1:6650]] Sending ping message
[localhost/127.0.0.1:6650] Received cmd PONG

Would like the log to just be:

15:23:59.459 [main] DEBUG com.pulsar_logging_producer.Producer -==========TEST MESSAGE=============

Consumer.java:

public class Consumer {

    org.apache.pulsar.client.api.Consumer<byte[]> consumer;
    PulsarClient client;
    private final String SERVICE_URL = "pulsar://localhost:6650";
    private static final Logger logger = LogManager.getLogger(Consumer.class);

    public Consumer(String topic, String subName) throws PulsarClientException {
        client = PulsarClient.builder()
                .serviceUrl(SERVICE_URL)
                .build();
        consumer = client.newConsumer()
                .topic(topic)
                .subscriptionName(subName)
                .subscribe();
    }

    public void listen() throws PulsarClientException {
        while (true) {
            Message msg = consumer.receive();
            logger.trace(new String(msg.getData()));

            try {
                System.out.printf("Message received: %s\n", new String(msg.getData()));
                consumer.acknowledge(msg);
            } catch (Exception e) {
                System.out.println("MESSAGE FAILED");
                consumer.negativeAcknowledge(msg);
            }
        }
    }

    public void close() {
        try {
            consumer.close();
            client.close();
        } catch (PulsarClientException e) {
            e.printStackTrace();
        }
    }
}

Producer.java:

public class Producer {

    private static final Logger logger = LogManager.getLogger(Producer.class);

    public static void main(String[] args) {

        String msg = "==========TEST MESSAGE=============";
        System.out.printf("Message sent: %s\n", msg);
        logger.debug(msg);
    }
}

log4j2.xml for Producer:

<Configuration status="WARN">
    <Properties>
        <Property name="DEFAULT_PATTERN" value="%d{HH:mm:ss.SSS} [%t] %-5level %logger{36} -%msg%n"/>
    </Properties>
    <Appenders>
        <Pulsar name="pulsarAppender" serviceUrl="pulsar://localhost:6650" topic="coinflex_logs" avoidRecursive="false">
            <PatternLayout pattern="${DEFAULT_PATTERN}"/>
        </Pulsar>
    </Appenders>
    <Loggers>
        <Logger name="org.apache.pulsar" level="INFO"/>
        <Root level="debug">
            <AppenderRef ref="pulsarAppender"/>
        </Root>
    </Loggers>
</Configuration>

score 0 · Answer 1 · answered Jun 26 '20 at 07:39

Turns out my problem was because I'm always defaulting to the root logger, so pulsar messages get logged there too. All I needed to do was make the root logger log to console and the Consumer logger to log to the rollingFileAppender:

<Configuration status="WARN">
    <Properties>
        <Property name="LOG_DIR" value="logs"/>
    </Properties>
    <Appenders>
        <Console name="consoleAppender" target="SYSTEM_OUT">
            <PatternLayout pattern="%msg%n"/>
        </Console>
        <RollingFile
            name="rollingFileAppender"
            fileName="${LOG_DIR}/coinflex_v2.log"
            filePattern="${LOG_DIR}/coinflex_v2_%i.log"
        >
            <PatternLayout pattern="%msg%n"/>
            <Policies>
                <SizeBasedTriggeringPolicy size="10MB"/>
            </Policies>
            <DefaultRolloverStrategy max="5"/>
        </RollingFile>
    </Appenders>
    <Loggers>
        <Root level="debug">
            <AppenderRef ref="consoleAppender"/>
        </Root>
        <Logger name="com.pulsar_logging_consumer.Consumer" level="debug" additivity="false">
            <AppenderRef ref="rollingFileAppender"/>
        </Logger>
    </Loggers>
</Configuration>

Using Pulsar as appender to Log4j2: How to stop logging acknowledgements?

1 Answers1