Adding Zipkin tracing to WildFly via CLI using startup scripts

Question

I am trying to add tracing on a Wildfly server (specifically Keycloak Docker image)

Following this document https://docs.wildfly.org/19/Admin_Guide.html#MicroProfile_OpenTracing_SmallRye

I got as far as

/extension=org.wildfly.extension.microprofile.opentracing-smallrye:add
/subsystem=microprofile-opentracing-smallrye:add

But I can't get the next parts working to set it to point to zipkin:9411

The next command in the instructions failed

[standalone@localhost:9990 /]  /subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:add()
{
    "outcome" => "failed",
    "failure-description" => "WFLYCTL0030: No resource definition is registered for address [
    (\"subsystem\" => \"microprofile-opentracing-smallrye\"),
    (\"jaeger-tracer\" => \"my-tracer\")
]",
    "rolled-back" => true
}

However, doing it using /opt/jboss/startup-scripts/ also fails

Executing cli script: /opt/jboss/startup-scripts/enable-tracing.cli
No connection to the controller.

Using @ehsavoie answer I got a bit further

embed-server --admin-only=true
/extension=org.wildfly.extension.microprofile.opentracing-smallrye:add()
/subsystem=microprofile-opentracing-smallrye:add()
/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:add()
/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:write-attribute(name=sender-endpoint,value=http://tracing:9411)
/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:write-attribute(name=propagation,value=[B3])
/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:write-attribute(name=reporter-log-spans,value=true)
/subsystem=microprofile-opentracing-smallrye:write-attribute(name=default-tracer,value=my-tracer)
stop-embedded-server

but still does not log to zipkin which uses B3.

I also tried

/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:write-attribute(name=sender-endpoint,value=http://tracing:9411/api/v1/spans)

I just tried your commands on WildFly 19 after removing the opentracing extension and subsystem and it passed. Could you please put a link to the image you are using ? — ehsavoie, May 04 '20 at 09:55

score 1 · Answer 1 · answered May 06 '20 at 13:43

1

embed-server --admin-only=true
/extension=org.wildfly.extension.microprofile.opentracing-smallrye:add()
/subsystem=microprofile-opentracing-smallrye:add()
/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:add()
stop-embedded-server

This jboss-cli script should enable opentracing before starting the server properly. I'm not sure how/when you can execute that with keycloack image

answered May 06 '20 at 13:43

ehsavoie

3,126
1
16
14

once you add that how do you configure it though? say I want the tracing to use zipkin on 192.168.1.1:9411 – Archimedes Trajano May 06 '20 at 23:23
Well https://wildfly.org/news/2020/03/19/Micro_Profile_OpenTracing_Comes_To_WildFly/ looks like something useful ;) – ehsavoie May 07 '20 at 09:55
I looked at your project, its kind of weird that i can't find where you set the jaeger servers or enable the tracing. – Archimedes Trajano May 07 '20 at 16:15
/subsystem=microprofile-opentracing-smallrye/jaeger-tracer=jaeger-demo:write-attribute(name="sender-endpoint", value="http://localhost:14268/api/traces") – ehsavoie May 07 '20 at 16:38
I was talking about your github project https://github.com/ehsavoie/opentracing-demo – Archimedes Trajano May 07 '20 at 20:30
a JAX-RS application is automatically instrumented for tracing, i don't know keycloack internals to knows what is instrumented. – ehsavoie May 08 '20 at 08:44

score 0 · Answer 2 · answered May 05 '20 at 09:19

You need to reload after adding the subsystem:

[standalone@localhost:9990 /] /extension=org.wildfly.extension.microprofile.opentracing-smallrye:add
{"outcome" => "success"}

[standalone@localhost:9990 /] /subsystem=microprofile-opentracing-smallrye:add
{
    "outcome" => "success",
    "response-headers" => {
        "operation-requires-reload" => true,
        "process-state" => "reload-required"
    }
}

[standalone@localhost:9990 /] reload
[standalone@localhost:9990 /] /subsystem=microprofile-opentracing-smallrye/jaeger-tracer=my-tracer:add()
{"outcome" => "success"}

can it be done from the a CLI startup script? as I had noted in the OP that I wanted it in startup scripts — Archimedes Trajano, May 05 '20 at 12:26

score 0 · Answer 3 · answered Oct 23 '21 at 19:10

I would recommend https://github.com/open-telemetry/opentelemetry-java-instrumentation for tracing. It needs to be started as javaagent, so JAVA_OPTS_APPEND can be used with official Keycloak Docker image, e.g.:

JAVA_OPTS_APPEND: "-javaagent:/opentelemetry-javaagent-all.jar"

And then agent can be configured via env variables, e.g.:

OTEL_SERVICE_NAME: keycloak
OTEL_TRACES_EXPORTER: jaeger
OTEL_EXPORTER_JAEGER_ENDPOINT: <jaeger endpoint>

It works also with future Keycloak X version and without complicated startup scripts.

Adding Zipkin tracing to WildFly via CLI using startup scripts

3 Answers3