0

I am trying to get open telemetry tracing setup in my spring boot 2.7 project. It is just a simple REST api project.

The tutorial I am following is the following one. I checked several other articles as well, but all of them discuss the same thing.

https://www.baeldung.com/spring-boot-opentelemetry-setup

I have added following dependencies in my pom.xml

<dependencies>
    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-web</artifactId>
    </dependency>

    <dependency>
        <groupId>org.springframework.boot</groupId>
        <artifactId>spring-boot-starter-test</artifactId>
        <scope>test</scope>
    </dependency>

    <!-- Sleuth with Brave tracer implementation -->
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-starter-sleuth</artifactId>
        <exclusions>
            <!-- Exclude Brave (the default) -->
            <exclusion>
                <groupId>org.springframework.cloud</groupId>
                <artifactId>spring-cloud-sleuth-brave</artifactId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- Add OpenTelemetry tracer -->
    <dependency>
        <groupId>org.springframework.cloud</groupId>
        <artifactId>spring-cloud-sleuth-otel-autoconfigure</artifactId>
    </dependency>

</dependencies>

I have following dependencyManagement tag as well in the pom file.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-dependencies</artifactId>
            <version>2021.0.5</version>
            <type>pom</type>
            <scope>import</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.cloud</groupId>
            <artifactId>spring-cloud-sleuth-otel-dependencies</artifactId>
            <version>1.1.1</version>
            <scope>import</scope>
            <type>pom</type>
        </dependency>
    </dependencies>
</dependencyManagement>

My application just exposes a simple endpoint that returns a simple string.

private final Logger logger = LoggerFactory.getLogger(GreetingsController.class);

@GetMapping("/hello-two")
public String helloFromService2() {
    logger.info("Hello from service 2");
    return "Hello from service 2";
}

Now the issue is, when I start the application and invoke the endpoint, I see a series of zeros as the trace id and the span id

2023-05-23 22:50:05.770  INFO [service-2,00000000000000000000000000000000,0000000000000000] 63434 --- [nio-9092-exec-5] c.e.s.controller.GreetingsController     : Hello from service 2

If I use brave without excluding it, the trace id and the span id start appearing. This gives me the impression that there is something that is unknown to me in spring-cloud-sleuth-otel-autoconfigure. Do I need to provide any configuration for that to inject the trace id and the span id ? Or what's happening here? How can I get this working.

Any help on this will be highly appreciated.

Thanks in advance.


EDIT: I am using opentelemetry java agent to collect the metrics of the application.

java -javaagent:./opentelemetry-javaagent-1.22.0.jar \
 -jar target/spring-tracing-0.0.1-SNAPSHOT.jar \
 --spring.config.location=./src/main/resources/application.properties
Jonatan Ivanov
  • 4,895
  • 2
  • 15
  • 30
vigamage
  • 1,975
  • 7
  • 48
  • 74
  • Could you please try to upgrade your dependencies spring-cloud and spring-cloud-sleuth-otel boms? Also, are you defining the OTel BOM/SDK version somewhere or do you get the OTel version from the sleuth-otel BOM? (This should not matter but you might also want to add an exporter) – Jonatan Ivanov May 24 '23 at 20:32
  • @JonatanIvanov I was able to resolve the issue. There was a one important point I have missed here in the question. I am using open telemetry java agent to run the app. Later realised that if I run w/o that, the application prints the trace and span id correctly. To resolve, I had to make sure I am using a javaagent version that supports the otel sdk version used by spring cloud in my project and then I had to use MDC for logging. check here `https://github.com/open-telemetry/opentelemetry-java-instrumentation/discussions/8572` Thanks @JonatanIvanov – vigamage May 25 '23 at 16:57
  • 1
    If you use the OTel agent, that has nothing to do with Spring Boot or Sleuth, OTel does everything, you are not using instrumentation provided by the Spring portfolio. – Jonatan Ivanov May 31 '23 at 19:11

1 Answers1

0

Keeping the solution added as a answer here.

To resolve this issue, I had to make sure that I am using a supported opentelemetry java agent version by spring cloud dependencies I am using. Internally spring was using version 1.22.0 of open telemetry SDK. So I switched to opentelemetry java agent version 1.22.0 from 1.26.0

Then I had to use MDC for logging. in application.properties file, I declared the following.

logging.pattern.level = [${spring.application.name:},%mdc{trace_id:-},%mdc{span_id:-}] %5p

After those two changes, it started working.

Note that, when I started the application without using opentelemetry java agent, even without the logging pattern declaration in property file, the application printed the trace id and the span id. It did not happen when I ran the application with otel java agent.

Thanks goes to opentelemetry-java-instrumentation community. checkout below link

https://github.com/open-telemetry/opentelemetry-java-instrumentation/discussions/8572

vigamage
  • 1,975
  • 7
  • 48
  • 74