2

Hi I am pretty new to apache kafka, I dont know how much sense this will make.

I did lot of research and couldn't find whats the advantage of multiple brokers. Went through the whole kafka documentation and couldn't find an answer for this.

Say for example I am receiving data from two different set of devices which I should manipulate and store.Depending on from which set of device data arrives the consumer will change. Should I go with multi broker - single topic - multi partition OR single broker - single topic - multi partition OR some other approach ??

Any help or guide is appreciated.

aladeen
  • 297
  • 7
  • 18

1 Answers1

2

As with pretty much any distributed system: scalability and resiliency. One broker goes down - no problem if you have replication set up. You suddenly get a traffic spike which would be too much for a single machine to handle - no problem if you have a cluster of machines to handle the traffic.

Lundahl
  • 6,434
  • 1
  • 35
  • 37
  • Thanks man. On a different note can I point a storm spout to a particular partition, as in read the data only from one single partition out of multiple partitions ??? – aladeen Apr 04 '16 at 13:08
  • 1
    I've not been following Storm very closely lately but I would be surprised if you could not achieve that. – Lundahl Apr 04 '16 at 16:13
  • Seems like we can just mention the number of parallelism for storm and that is like multi threading (or exactly multi threading), and doesnt allow to point to one particular partition. – aladeen Apr 05 '16 at 05:12