Elastic APM + Spring + @KafkaListener - @CaptureSpan not working

Question

I have a simple spring boot + kafka + apm app. I use ELK+APM Server 7.12.1 version, java agent co.elastic.apm:apm-agent-attach:1.25.0 and agent api co.elastic.apm:apm-agent-api:1.28.3

My Elastic apm configuration looks like this:

@Setter
@Configuration
@ConfigurationProperties(prefix = "app.elastic-apm")
public class ElasticApmConfiguration {

    private static final String SERVER_URL_KEY = "server_url";
    private String serverUrl;

    private static final String SERVICE_NAME_KEY = "service_name";
    private String serviceName;

    private static final String ENVIRONMENT_KEY = "environment";
    private String environment;

    private static final String APPLICATION_PACKAGES_KEY = "application_packages";
    private String applicationPackages;

    @PostConstruct
    public void init() {
        Map<String, String> apmProps = new HashMap<>();
        apmProps.put(SERVER_URL_KEY, serverUrl);
        apmProps.put(SERVICE_NAME_KEY, serviceName);
        apmProps.put(ENVIRONMENT_KEY, environment);
        apmProps.put(APPLICATION_PACKAGES_KEY, applicationPackages);

        ElasticApmAttacher.attach(apmProps);
    }
}

I have a @KafkaListener like this:

@Slf4j
@EnableKafka
@Component
@RequiredArgsConstructor
public class MyListener {

    ...deps...

    @KafkaListener(topics = "${app.kafka.topics.mytopic}")
    public void consume(@Payload String input, Acknowledgment acknowledgment) {
        ... my logic ...
        methodIWouldLikeToProfile();
        .....
        acknowledgment.acknowledge();
    }

    private String methodIWouldLikeToProfile(){
        final var span = ElasticApm.currentSpan().startSpan();
        span.setName("calculateTotalQuantity");
        System.out.format("CALCULATING QUANTITY\n");
        ...some more logic....
        span.end();
        return res;
    }
}

Everything works smoothly I see in my kibana the span named calculateTotalQuantity However when I try to use the @CaptureSpan annotation it stops working

@CaptureSpan(value = "calculateTotalQuantity")
private String methodIWouldLikeToProfile(){
        System.out.format("CALCULATING QUANTITY\n");
        ...some more logic....
        return res;
    }

and I don't see the span anymore. It can be seen even in the logs:

2021-12-29 14:49:42,131 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.Span - startSpan '' 00-2564fc689dc11bfff90c3501c0c50a13-e508218c002de7c7-01 (2b378884)
2021-12-29 14:49:42,133 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'Kafka record from mytopic.on.kafka' 00-2564fc689dc11bfff90c3501c0c50a13-e0088731afcf4284-01 (323a0459) (4)
2021-12-29 14:49:42,136 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to '' 00-2564fc689dc11bfff90c3501c0c50a13-e508218c002de7c7-01 (2b378884) (1)
2021-12-29 14:49:42,136 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'Kafka record from mytopic.on.kafka' 00-2564fc689dc11bfff90c3501c0c50a13-e0088731afcf4284-01 (323a0459) (5)
2021-12-29 14:49:42,137 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to 'Kafka record from mytopic.on.kafka' 00-2564fc689dc11bfff90c3501c0c50a13-e0088731afcf4284-01 (323a0459) (6)
2021-12-29 14:49:42,137 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - increment references to '' 00-2564fc689dc11bfff90c3501c0c50a13-e508218c002de7c7-01 (2b378884) (2)
CALCULATING QUANTITY
2021-12-29 14:49:42,150 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.Span - endSpan 'calculateTotalQuantity' 00-2564fc689dc11bfff90c3501c0c50a13-e508218c002de7c7-01 (2b378884)
2021-12-29 14:49:42,151 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - decrement references to 'Kafka record from mytopic.on.kafka' 00-2564fc689dc11bfff90c3501c0c50a13-e0088731afcf4284-01 (323a0459) (5)

with the annotation:

2021-12-29 14:47:22,813 [org.springframework.kafka.KafkaListenerEndpointContainer#0-0-C-1] DEBUG co.elastic.apm.agent.impl.transaction.AbstractSpan - decrement references to 'Kafka record from mytopic.on.kafka' 00-920621868563cf9cd9ad2a5cc89e35a8-cb6aa4ca6a00f907-01 (597b1ec5) (3)
2021-12-29 14:47:22.816  INFO 90644 --- [ntainer#0-0-C-1] e.v.s.s.s.d.service.MyService     : Did something
CALCULATING QUANTITY
2021-12-29 14:47:22.821  INFO 90644 --- [ntainer#0-0-C-1] .s.s.MyListener : Did something elese
2021-12-29 14:47:22.822  INFO 90644 --- [ntainer#0-0-C-1] .s.s.MyListener : And else

Could someone tell me how to solve it ?

score 0 · Answer 1 · answered Nov 29 '22 at 15:59

Not sure if this is still relevant for you, I've encountered the same problem where elastic.apm.application_packages Java system property was not loaded properly from my application.properties file.

As per documentation (https://www.elastic.co/guide/en/apm/agent/java/1.x/config-stacktrace.html#config-application-packages):

You can set the application package using Java system properties: -Delastic.apm.application_packages=com.myproject

Just to make sure this has been correctly loaded, add the following line within IntelliJ VM options application Environment configuration:

-javaagent:elastic-apm-agent-1.34.1.jar -Delastic.apm.application_packages=com.myproject

then check your application startup logs, it should contain a log line like below:

2022-11-29 17:24:13,274 [main] INFO  co.elastic.apm.agent.configuration.StartupInfo - application_packages: 'com.myproject' (source: Java System Properties)

Cheers!

Elastic APM + Spring + @KafkaListener - @CaptureSpan not working

1 Answers1