0

Is Apache Helix capable of rebalancing data between nodes? For examle I'm developing an in-memory database where some of the partitioned/sharded data needs to be moved to another node. Is messaging or the Application Property Store a way to solve this problem or is it necessary to find another solution for data streaming between nodes?

Regarding the latter is it comfortable and efficient to use MessageQueues/Systems like RabbitMQ or Apache Kafka?

Towen
  • 175
  • 1
  • 1
  • 10
  • What type of in memory database? What kinds of data are in it? – OneCricketeer Oct 20 '18 at 19:06
  • @cricket_007 just arbitrary data. It could be any kind of state. The focus is on the transfer. I'm developing a service with state which gets partitioned and replicated. The in memory database was just an example. – Towen Oct 20 '18 at 22:05

1 Answers1

0

Disclaimer: I don't know Helix or Rabbit

If you used Kafka, it would be persistent on servers rather than in memory, but you would need to write separate consumers to rebuild said database elsewhere.

For example, Kafka Stream's KTable object are exactly for that purpose. The partitioning of your topics would be up to you to implement if not using the default behaviour.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245