1

When a consumer fails to process a record, the SeekToCurrentErrorHandler kicks in and does its job.

After 10 retries, I can see a log line that the backoff was exhausted for the erroring record, however the logged record now has no headers.

On further inspection, I can see that all retries of the record are correctly linked to the overall Trace, however the logs from KafkaMessageListenerContainer$ListenerConsumer after each failure, and then the log line from SeekToCurrentErrorHandler, are not trace aware.

How can I go about fixing this? Should I pass a custom error handler to SeekToCurrentErrorHandler with my own logger in the hopes that it is trace aware?

Log lines missing the trace:

KafkaMessageListenerContainer$ListenerConsumer [ERROR] Error handler threw an exception
SeekToCurrentErrorHandler [ERROR] Backoff FixedBackOff{...} exhausted for ConsumerRecord(topic=..., ..., headers = RecordHeaders(headers = [], ...), ...)

I have removed some data from the above log lines for the sake of brevity.

Edit: Adding versions

spring-cloud-starter-sleuth: 2.2.4.RELEASE spring-kafka: 2.5.5.RELEASE

Rhys Saldanha
  • 21
  • 3
  • Which version of Sleuth are you using? We can't help you without this information. Please ensure that you're using the latest version. If that's the case or if you still see issues you can file an issue in the Brave Github repository cause Sleuth is just referencing Brave Kafka's tracing. – Marcin Grzejszczak Oct 13 '20 at 10:35
  • Excuse me, and what is an expected behavior? What you would like to see in those logs? – Artem Bilan Oct 13 '20 at 14:17
  • @ArtemBilan the expected behaviour is that the trace appears in these log messages - instead I am seeing `[,]` showing that no trace + span were associated with the logs – Rhys Saldanha Oct 17 '20 at 16:07
  • Right. How that should look? The point is that I’m not familiar with all of these tracing features, but I may help with the fix in Spring Kafka. So, it might even be great if you can share a simple project to play with including an explanation what to expect in logs. You teach me with traces and I’ll help you with Spring for Apache Kafka – Artem Bilan Oct 17 '20 at 16:13

0 Answers0