0

I want to build a trident topology which will get tuples from Kafka, group them by one field and persist entire grouped tuples in HBase. Reason for grouping is that HBase put operations are faster with multiple put objects compared to putting single object, I want to group Put objects by one field and insert all those Put objects in single HBase API call.

I have written following code but not sure where should I write code which will convert all tuples into HBase 'Put' type objects and persist all those Put objects in HBase.

OpaqueTridentKafkaSpout kafkaTridentSpout = new OpaqueTridentKafkaSpout(spoutConfig);
TridentTopology topology = new TridentTopology();
topology.newStream("stream", kafkaTridentSpout)
                             .groupBy(new Fields("pointId"))
                             .toStream();
topology.build();

Can anyone please help me?

Matthias J. Sax
  • 59,682
  • 7
  • 117
  • 137
Shekhar
  • 11,438
  • 36
  • 130
  • 186

1 Answers1

-1

Storm has bolts and spouts. Spouts listen to sources, and bolts are on the other side, working in parallel and moving the data to destination. In your case, may be the spout should be listening to kafka, get the data in a Bolt and bolt should write to Hbase

Ramzy
  • 6,948
  • 6
  • 18
  • 30