0

i am going to build stream processing systems. Using Kafka for message transmission. Stream processing can be done with storm trident, storm or spark streaming. But couldn't find the best answer.

Stream Processing will be simple if conditions at the beginning. Field can be inside different tuple, that's why i need to store previous field. Now i implemented this scenario by using storm + redis (not trident). Inside bolt, get the all fields from redis, then measuring by if conditions. If fields not null then use, if null continue working. I have bad feelings about architecture if this is correct.

Should i use trident api or spark streaming?

I don't need "exactly once" delivery guarantees. Just need state of each data source. This state will be calculated by fields.

Thanks for reply.

Guy Korland
  • 9,139
  • 14
  • 59
  • 106
Joker
  • 96
  • 6
  • You should check out Kafka Streams for your processing: http://kafka.apache.org/documentation/streams and http://docs.confluent.io/current/streams/index.html – Matthias J. Sax Mar 27 '17 at 21:23
  • Should i use BaseStatefulBolt? https://github.com/apache/storm/blob/master/examples/storm-starter/src/jvm/storm/starter/StatefulTopology.java – Joker Mar 28 '17 at 07:02
  • @Joker did you try https://github.com/RedisLabs/spark-redis? – Guy Korland Oct 25 '18 at 08:52

0 Answers0