2

I am running a Kafka Streams application in Spring Boot using the Spring Cloud Stream binder. The application has several instances with every instance having 4 threads (num.stream.threads=4). However when navigating to the Actuator health endpoint (/actuator/health) I see details of only 1 thread in each instance. Is that an expected behaviour or am I doing something wrong?

Does that mean that when using InteractiveQueryService in a REST endpoint, the service would query only state stores belonging to the single thread and the remaining 3 threads in the instance are inaccessible?

Update:

The kstream part of the health result looks like this:

kstream: {
  status: "UP",
  details: {
    streams.job-part-index-v1-1-20230405-1: {
      adminClientId: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-admin",
      restoreConsumerClientId: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4-restore- consumer",
      threadState: "RUNNING",
      producerClientIds: [
        "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef- b1b6-d249546b53ab-StreamThread-4-producer"
      ],
      standbyTasks: { },
      activeTasks: {
      partitions: [
        "partition=13, topic=integration.access-control-job-part-v1-0",
        "partition=13, topic=integration.job-part-v1-0",
        "partition=13, topic=streams.job-part-index-v1-1-20230405-1-COGROUPKSTREAM-AGGREGATE-STATE-STORE-0000000006-repartition"
      ],
      taskId: {
        topicGroupId: 1,
        partition: 13
      }
    },
    consumerClientId: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4-consumer",
    threadName: "streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4"
  }
}

Meanwhile all the threads in the instance are busy as visible in the log:

{"timestamp":"2023-05-22 06:56:57.654","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-1","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-1] Processed 262 total records, ran 8 punctuators, and committed 29 total tasks since the last update","context":"default"}
{"timestamp":"2023-05-22 06:57:29.502","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-2","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-2] Processed 280 total records, ran 4 punctuators, and committed 92 total tasks since the last update","context":"default"}
{"timestamp":"2023-05-22 06:58:15.243","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-3","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-3] Processed 78 total records, ran 0 punctuators, and committed 67 total tasks since the last update","context":"default"}
{"timestamp":"2023-05-22 06:58:15.628","level":"INFO","thread":"streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4","logger":"org.apache.kafka.streams.processor.internals.StreamThread","message":"stream-thread [streams.job-part-index-v1-1-20230405-1-7a89107e-6487-47ef-b1b6-d249546b53ab-StreamThread-4] Processed 81 total records, ran 0 punctuators, and committed 72 total tasks since the last update","context":"default"}

The number of input partitions is higher than the total number of threads.

The application uses Spring Boot 2.6.6 and Spring Cloud Stream 3.2.1

user2732824
  • 141
  • 1
  • 5
  • If you have four threads per instance, all of them should report health status. The binder queries the metadata for all the local threads per instance and then reports that up to the health endpoint. – sobychacko May 19 '23 at 22:25
  • You should ensure that Kafka Streams create all threads. The input topic needs at least four partitions for the threads to become active. – sobychacko May 19 '23 at 22:26
  • `InteractiveQuerySerive` does not differentiate data per thread. It queries the state store by name and this is across the instance (multiple threads possibly). – sobychacko May 19 '23 at 22:27
  • @sobychacko All the threads are created as visible in the application log, but only 1 thread detail is described in the health endpoint. I added the details to the original post. – user2732824 May 22 '23 at 07:15

0 Answers0