1

My understanding was that spring-kafka was created to interact with Kafka Client APIs, and later on, spring-cloud-stream project was created for "building highly scalable event-driven microservices connected with shared messaging systems", and this project includes a couple of binders, one of them is a binder that allows the interaction with Kafka Stream API:

spring-cloud-stream-binder-kafka-streams

So it was clear to me that if I want to interact with Kafka Stream API, I will use the spring-cloud-stream approach with the appropriate binder.

But, I found out that you can interact with Kafka Stream API also with the spring-kafka approach.
Need the below two dependencies. One example is here.

'org.springframework.kafka:spring-kafka'
'org.apache.kafka:kafka-streams'

So my question is - if both the approaches allow interaction with Kafka Stream API, what are the differences between the approaches?

YevgenyL
  • 281
  • 3
  • 20
  • 1
    Spring Cloud Stream is a higher level of abstraction (both binders use spring-kafka underneath). It provides opinionated configuration defaults, auto-provisions topics etc. And, aside from when using Kafka Streams, it allows you to easily switch transports (e.g. to RabbitMQ instead of Kafka). When using Kafka Streams, it's not so flexible since you are still using the Streams API. – Gary Russell Nov 10 '21 at 14:06

1 Answers1

1

As Gary pointed out in the comments above, spring-kafka is the lower-level library that provides the building blocks for Spring Cloud Stream Kafka Streams binder (spring-cloud-stream-binder-kafka-streams). The binder provides a programming model to write your Kafka Streams processor as a java.util.function.Function or java.util.function.Consumer.

You can have multiple such functions and each of them will build its own Kafka Streams topologies. Behind the scenes, the binder uses Spring-Kafka to build the Kafka Streams StreamsBuilder object using the StreamsBuilderFactoryBean. Binder also allows you to compose various functions. The functional model comes largely from Spring Cloud Function, but it is adapted for the Kafka Streams in the binder implementation. The short answer is that both spring-Kafka and the Spring Cloud Stream Kafka Streams binder will work, but the binder gives a programming model and extra features consistent with Spring Cloud Stream whereas spring-kafka gives various low-level building blocks.

Diogo Moreira
  • 1,082
  • 2
  • 9
  • 24
sobychacko
  • 5,099
  • 15
  • 26