How to implement trace service to service using spring webflux?

Question

I am implementing service to service integration that using spring webflux. Each microservice is isolated and running a different port. I would like to see an end to end trace using jaeger. The problem is each service is capturing trace without issue but I can't see service to service communication and architectural design.

Preference-service is receiving the request and forwarding customer-service.

Using sleuth and zipkin for collecting trace

implementation 'org.springframework.boot:spring-boot-starter-webflux'
implementation 'org.springframework.cloud:spring-cloud-sleuth-zipkin'
implementation 'org.springframework.cloud:spring-cloud-starter-sleuth'

Preference service code part

   @RequestMapping("/preference")
@ContinueSpan
public Flux<?> preference(@RequestHeader Map<String, String> headers) {
   
    return callByPriceReactive( headers);
}

@NewSpan("chainingPriceReactive")
private Flux<?> callByPriceReactive(Map<String, String> headers ){
    logger.debug("starting statement");
    logger.info("active span: " + tracer.activeSpan());

    return webClient.get().uri("/customerByPrice?maxPrice=1")
           // .header("X-B3-TRACEID", headers.get("X-B3-TRACEID"))
            .retrieve()
            .bodyToFlux(String.class)
            .checkpoint();


}

Preference service prop file

logging.level.com.trace.customer=debug
server.port=7072
logging.level.org.springframework.web=info
spring.sleuth.traceId128=true
spring.sleuth.sampler.probability=1.0
spring.application.name=preference-application 
customer.api.url= http://localhost:7071
spring.sleuth.opentracing.enabled=true
##spring.zipkin.base-url=http://<your-Jaeger-server>:<port>

Customer-service code part

  @RequestMapping("/customerByPrice")
@ContinueSpan
public ResponseEntity<Flux<?>> getCustomer(@RequestParam int maxPrice,@RequestHeader Map<String, String> headers) {
    
    logger.debug("tracer: " + tracer);
    logger.info("active span: " + tracer.activeSpan());
    // just a log statement to show the current context
    logger.debug("starting statement");

    return ResponseEntity.ok(customerService.
            generateRandomCustomer(maxPrice)
          //  .delayElements(Duration.ofSeconds(1))
            .doOnNext(customer -> logger.info("Found customer  {} for ${}", customer.getCustomerId(), 
             customer.getAmount()))
            .doOnComplete(() -> logger.debug("done!"))
            .doOnError(e -> logger.error("failure", e)));
}

Customer service prop file

logging.level.com.trace.customer=debug
server.port=7071
logging.level.org.springframework.web=info 
spring.sleuth.traceId128=true
spring.sleuth.sampler.probability=1.0
spring.application.name=customer-application 
spring.sleuth.opentracing.enabled=true
##spring.zipkin.base-url=http://<your-Jaeger-server>:<port>

Jaeger is running in the docker-compose file

version: "3"

services:
 jaeger:
  image: "jaegertracing/all-in-one:latest"
  environment:
   - COLLECTOR_ZIPKIN_HTTP_PORT=9411
  ports:
  - "5775:5775/udp"
  - "6831:6831/udp"
  - "6832:6832/udp"
  - "5778:5778"
  - "16686:16686"
  - "14268:14268"
  - "9411:9411"

Preference service trace in Jaeger Customer service trace in Jaeger

Answer 1

Based on this:

The problem is each service is capturing trace without issue but I can't see service to service communication and architectural design.

it seems that the tracing information is not propagated across services. You can check this by looking into the HTTP headers and check the traceId. In order to make this work the traceId should be the same across the requests. You should see the same traceId in the logs too.

The documentation gives you some pointers how to troubleshoot this:

• How to Make RestTemplate, WebClient, etc. Work?
• WebClient integration