Using OTEL Java agent, how to create a new Context without using @WithSpan

Question

The opentelemetry-javaagent-all agent (versions 0.17.0 and 1.0.1) has been the starting point for adding trace information to my Java application. Auto-instrumentation works great.

Some of my application cannot be auto-instrumented. For this part of the application, I began by adding @WithSpan annotations to interesting spots in the code.

I now reach the limits of what seems possible with simple @WithSpan annotations. However, the framework underlying my app allows me to register callbacks to be invoked at certain points -- e.g. I can provide handlers that are notified when a client connects / disconnects.

What I think I need is to start a new Span when Foo.onConnect() is called, and set it be the parent for the Spans that correspond to each request.

public class Foo {

    void onConnect() {
        // called when a client connects to my app
        // Here I want to create a Span that will be the parent of the Span created in
        // Foo.processEachRequest().
    }

    @WithSpan
    public void processEachRequest() {
        // works, but since it is called for each request... each span is in a separate Trace
    }

    void onDisconnect() {
        // called when the client disconnects from my app
        // Here I can end the parent Span.
    }
}

Other ideas - that didn't work out:

1 - The obvious solution would be to add @WithSpan annotations to the underlying framework. For various reasons, this is not going to be a practical way forward.

2 - Next choice might be to search for a way to tell the javaagent about methods in my underlying framework. (The New Relic agent can do something like this.) That doesn't seem to be a feature of the open-telemetry agent, today anyway.

So, I'm left with looking for a way to do this using the callbacks, as above. Is there a way to do this?

If you need a span on `onConnect`, why not annotate the `onConnect` method with `@WithSpan`? — knittl, Mar 22 '21 at 18:33
The annotation would just create a Span for the duration of the onConnect method.(The Span would end when the onConnect returned.) — Paul S, Mar 22 '21 at 19:47
One more question: how long is a client usually connected? You would usually want to avoid creating a span which is open for minutes or hours. How can you identify the client in the `processEachRequest()` method? — knittl, Apr 27 '21 at 18:49

score 1 · Answer 1 · answered Apr 27 '21 at 18:48

That should be possible by manually instrumenting your code. You would use the Tracer interface of OpenTelemetry, as described in the OpenTelemetry Java docs.

This should give you a general idea:

public class Foo {
    private Span parentSpan; // you might need a Map/List/Stack here

    void onConnect() {
        Tracer tracer =
                openTelemetry.getTracer("instrumentation-library-name", "1.0.0");
        Span span = tracer.spanBuilder("my span").startSpan();
        this.parentSpan = span; // might need to store span per request/client/connection-id
    }

    public void processEachRequest() {
        final Span parent = this.lookupParentSpan();
        if (parent != null) {
            try (Scope scope = span.makeCurrent()) {
              yourLogic();
            } catch (Throwable t) {
              span.setStatus(StatusCode.ERROR, "error message");
              throw t;
            }
        } else {
            yourLogic();
        }
    }

    void onDisconnect() {
        final Span parent = this.lookupParentSpan();
        if (parent != null) {
            parent.end();
        }
    }

    private Span lookupParentSpan() {
        // you probably want to lookup the span by client or connection id from a (weak) map
        return this.parentSpan;
    }
}

NB: You must guarantee that a span is always ended and does not leak. Make sure to properly scope your spans and eventually call Span#end().

Using OTEL Java agent, how to create a new Context without using @WithSpan

1 Answers1