my goal is to maintain the traceId (of a brave.Span) between a rabbitMq sender and consumer. In order to achieve this I use spring-cloud-sleuth autoconfigured in version 2.1.0.RELEASE
When creating a span and eventually sending a message over RabbitTemplate to the broker and receiving it in the same (test) application I would expect spring-cloud-sleuth to maintain the traceId between sender and consumer.
My observation is that when sending a Message over RabbitMq the traceId is not correctly appended to the headers of the message. There is indeed a traceId/context appended and submitted, but it is a new one, different from that I read when creating the span (see code below).
On the consumer the newly created (but unrelated) traceId then is correctly processed and can be read from the Tracer.
@Test
public void messaging_ShouldPreserveTraceId() {
final Span spanProbe = tracer.newTrace().name("TraceIT").start();
spanProbe.start();
final String traceIdBefore = spanProbe.context().traceIdString();
log.info("TRACE CONTEXT BEFORE: " + spanProbe.context());
log.info("TRACE ID BEFORE: " + traceIdBefore);
log.info("TRACE ID BEFORE Parent: " + spanProbe.context().parentIdString());
// send - the actual rabbitTemplate Call (in the producer) happens in the same (main) thread
try{
producer.sendAsString(ROUTING_KEY, "CONTENT");
}finally {
spanProbe.finish();
}
// consume
Awaitility.await().atMost(TEN_MINUTES).until(() -> {
assertThat(consumer.spans(), hasSize(1));
});
// assert
final Span consumerSpan = consumer.spans().get(0);
final String traceIdAfter = consumerSpan.context().traceIdString();
log.info("TRACE CONTEXT AFTER: " + consumerSpan.context());
log.info("TRACE ID AFTER: " + traceIdAfter);
assertEquals(traceIdAfter, traceIdBefore);
}
The consumer (test class field) is:
static class TraceTestListener implements MessageListener {
private final List<Span> spans = new ArrayList<>();
@Autowired
private Tracing tracing;
@Override
public void onMessage(Message message) {
log.info("---> Received MESSAGE: {}", message);
spans.add(tracing.tracer().currentSpan());
}
public List<Span> spans() {
return spans;
}
}
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(114) - TRACE CONTEXT BEFORE: 23ca5b3b9f068716/23ca5b3b9f068716
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(115) - TRACE ID BEFORE: 23ca5b3b9f068716
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(116) - TRACE ID BEFORE Parent: null
[Rabbit-4 ] org.springframework.amqp.rabbit.listener.BlockingQueueConsumer$InternalConsumer.handleDelivery(897) - Storing delivery for consumerTag: 'amq.ctag-HnuJEiuRHAHTYfzypJDW6w' with deliveryTag: '1' in Consumer@6c27e700: tags=[[amq.ctag-HnuJEiuRHAHTYfzypJDW6w]], channel=Cached Rabbit Channel: AMQChannel(amqp://asdasdaa@35.243.142.228:5672/asdasdaa,2), conn: Proxy@33ebe4f0 Shared Rabbit Connection: SimpleConnection@3a88f6fb [delegate=amqp://asdasdaa@35.243.142.228:5672/asdasdaa, localPort= 58539], acknowledgeMode=AUTO local queue size=0
// Please mind how in the received message's headers a different traceId is present
[test_rabbitConsumer1] org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.handle(469) - Received message: (Body:'[B@1989ac6d(byte[7])' MessageProperties [headers={X-B3-SpanId=b335bbaf06a08879, X-B3-Sampled=0, X-B3-TraceId=b335bbaf06a08879}, timestamp=Tue May 21 13:44:57 CEST 2019, contentType=text/plain; charset=utf-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=test-exchange, receivedRoutingKey=test-routing, deliveryTag=1, consumerTag=amq.ctag-HnuJEiuRHAHTYfzypJDW6w, consumerQueue=test-queue])
[test_rabbitConsumer1] org.springframework.cloud.sleuth.log.Slf4jScopeDecorator.log(180) - Starting scope for span: b335bbaf06a08879/46a25dd87dc63878
[test_rabbitConsumer1] org.springframework.cloud.sleuth.log.Slf4jScopeDecorator.decorateScope(102) - With parent: 4663306299116113188
[test_rabbitConsumer1] sl.euth.debug.boot.rabbit.trace.TraceIT$TraceTestListener.onMessage(150) - ---> Received MESSAGE: (Body:'[B@1989ac6d(byte[7])' MessageProperties [headers={}, timestamp=Tue May 21 13:44:57 CEST 2019, contentType=text/plain; charset=utf-8, contentLength=0, receivedDeliveryMode=PERSISTENT, priority=0, redelivered=false, receivedExchange=test-exchange, receivedRoutingKey=test-routing, deliveryTag=1, consumerTag=amq.ctag-HnuJEiuRHAHTYfzypJDW6w, consumerQueue=test-queue])
[test_rabbitConsumer1] org.springframework.cloud.sleuth.log.Slf4jScopeDecorator.log(180) - Closing scope for span: b335bbaf06a08879/46a25dd87dc63878
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(133) - TRACE CONTEXT AFTER: b335bbaf06a08879/46a25dd87dc63878
// Here we would expect the traceId to be 23ca5b3b9f068716
[main ] sl.euth.debug.boot.rabbit.trace.TraceIT.messaging_ShouldPreserveTraceId(134) - TRACE ID AFTER: b335bbaf06a08879
[main ] sl.euth.debug.commons.lib.test.junit.rules.LoggingRule$1.evaluate(77) - Finished test messaging_ShouldPreserveTraceId(sl.euth.debug.boot.rabbit.trace.TraceIT) in 13102 ms
