0

Are Kafka stream/redis stream good for reactive architecture? I am asking this mainly because both redis and kafka seem to be blocking threads when consuming the messages.

Are there any reasons behind this? I was hoping that I could read messages with some callback - so execute when the message was delivered like pub/sub in a reactive manner. Not by blocking thread.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
RamPrakash
  • 2,218
  • 3
  • 25
  • 54
  • The only blocking that is done is waiting for messages. It is optional for actual processing – OneCricketeer Jun 29 '21 at 21:43
  • 1
    Actually the recommendation (as far as I am aware of) is to process the messages on 'different thread' if your business case allows it.. (no ordering is needed and alike...) – Ran Lupovich Jun 29 '21 at 21:53

1 Answers1

0

Kafka client is relatively low level, what is "good" as in: it provides you with much flexibility when (and in which thread) you'd do the record processing. In the end, to receive a record, someone needs to block (as actual reading is sending fetch requests over and over). Whether the thread being blocked is a "main-business" thread or some side-i/o dedicated one will depend on developer's choice.

You might take a look at higher-level offerings such as Spring-Kafka or Kafka Stream API / Kafka Connect frameworks that provide "fatter" inversion-of-control containers, effectively answering the above concern.

Adam Kotwasinski
  • 4,377
  • 3
  • 17
  • 40