0

I am upgrading an application with Kotlin, Webflux to Spring Boot 3.

Now I noticed that our logs are lacking traceIds.

My suspicion is that this is due to coroutines, since I observed that the logs of the controller contain a traceId and in the logs of the service it is not included. The controller method as well as the service method both have the suspend keyword.

@PostMapping("/upload-file")
@Observed
suspend fun uploadFile(
    serverWebExchange: ServerWebExchange,
): ResponseEntity<MessageResponse> {
    logger.debug("uploadFile")
    importService.handle(getMultipartDataPart("file", serverWebExchange))
    return ResponseEntity(MessageResponse("Success."), OK)
}

@Service
class ImportService() {

suspend fun handle(fileAsByteArray: ByteArray) {
    log.debug("Import file.")
    ...
}

I followed the documentation and added depdencies

implementation("org.springframework.boot:spring-boot-starter-actuator")
implementation("io.micrometer:micrometer-tracing-bridge-brave")

Bean

@Bean
fun observedAspect(observationRegistry: ObservationRegistry?): ObservedAspect? {
    return ObservedAspect(observationRegistry)
}

and Annotation on the controller method

@GetMapping("/keys/{key}")
@Observed
suspend fun isKeyInKeyStore(

Another interesting observation I made is that I don't observe the problem on my local machine, but only on our dev K8s cluster. Could a different JVM (OpenJDK vs AWS Corretto) cause this?

I am quite sure that the context is lost, but not sure how to deal with it in the best way since I am pretty new to coroutines and Webflux.

Joachim Sauer
  • 302,674
  • 57
  • 556
  • 614
Nils Rommelfanger
  • 626
  • 7
  • 16
  • 3
    I don't know too much about Webflux, but you are probably correct. If Spring stores this info using `ThreadLocal` or similar, then it won't work with coroutines without additional work. Coroutines could freely switch from one thread to another. – broot Jan 23 '23 at 15:11
  • 1
    You can try to use this: https://kotlinlang.org/docs/coroutine-context-and-dispatchers.html#thread-local-data Although, depending on what the service does with coroutines, it could, but doesn't have to work. Also, for this you need access to the `ThreadLocal` instance. If it is kept internal or you don't know where it is stored, this won't help you. – broot Jan 23 '23 at 15:15
  • I forgot to mention that I found this https://betterprogramming.pub/tracing-in-spring-boot-3-webflux-d432d0c78d3e Under paragraph 3 there seems to be a proposed solution and I didn't successfully try it yet. I wonder if this is the only and recommended solution if it works. It seems pretty invasive for me to just propagate the MDC. – Nils Rommelfanger Jan 23 '23 at 15:43
  • 1
    I don't know Micrometer to fully understand this code, but it shouldn't require to wrap all logs. The proper solution is by using the [ThreadContextElement](https://kotlinlang.org/api/kotlinx.coroutines/kotlinx-coroutines-core/kotlinx.coroutines/-thread-context-element/) which allows to install hooks when switching to/from thread. Maybe there is an existing solution for Micrometer, otherwise it should be easy to implement. It doesn't provide 100% coverage for any coroutine code as the code can switch not only threads, but coroutine contexts as well, but it should work in most cases. – broot Jan 23 '23 at 19:43

0 Answers0