0

Scenario:

I have a north bound web service(say A) and a south bound application(say C). And I am creating a micro service (say B) which transforms data received by A in a format that is readable by C.

A can send data any random time intervals, and as B receives it, it has to transform.

What I think : Once B subscribes to A, and A starts sending data via a callback url. Save the data in mongodb and after some interval process the data and push to C.

Question: 1. Since A is streaming a type of data to B, Can I use KAFKA for consuming the data ? 2. If no, what are the other alternatives? 3. I want to know, is there any other efficient way of doing this ?

Milind Pandey
  • 13
  • 4
  • Kafka doesn't consume. Your app would. After another app pushes to it. For a single use case, Kafka is probably too complicated if all you want is a message bus – OneCricketeer Jul 12 '18 at 13:58

1 Answers1

0

What you are describing sounds to me like a typical stream transformation application. You can definitely use Kafka for this, but you can probably also use any other pub/sub service out there.

Setting up Kafka just for this seems like overkill and I would rather look into simpler pub/sub services offered online.

Marius Waldal
  • 9,537
  • 4
  • 30
  • 44
  • 1
    Thank you for the responding. I am also exploring rabbitMQ ? Can you suggest any sub services that might help me out. Preferably opensource. – Milind Pandey Jul 13 '18 at 09:54
  • Are you looking to host and run this yourself? Since you prefer it to be open source... These days, I'm reluctant to host/run anything myself unless it is an absolutely central part of what I'm building. There are many services out there that are both cheap and run by someone else, that'll just give you what you need without having to install, configure, administer etc yourself. Have a look at Google Cloud pub/sub for example. – Marius Waldal Jul 13 '18 at 10:08