1

How can I emit to multiple streams from the same bolt in Storm Trident?

I have a bolt which does some calculation and based on the result I want to pass some values to one stream, and some other values to another stream.

In Storm (not Trident), we could achieve that by following:

Split the stream into multiple streams:

@Override
public void declareOutputFields(final OutputFieldsDeclarer outputFieldsDeclarer) {
    outputFieldsDeclarer.declareStream("type1-stream", new Fields("type1"));
    outputFieldsDeclarer.declareStream("type2-stream", new Fields("type2"));
    outputFieldsDeclarer.declareStream("error-stream", new Fields("error"));
}

Then emit based on the findings, like: 

collector.emit("type1-stream", new Values("type 1 data"));
collector.emit("type2-stream", new Values("type 2 data"));
collector.emit("error-stream", new Values("error data"));

Then do the rest of the work by listening the expected stream:

builder.setBolt("errorBolt", errorBolt).shuffleGrouping("errorBoltStream", "error-stream");
builder.setBolt("type1Bolt", type1Bolt).shuffleGrouping("type1BoltStream", "type1-stream");

So how can I achieve the same behavior using Storm Trident?

One option is calling "each" for the same stream and run the same bolt and only emit based on what I want to emit to that stream or another option would be emit key and value pair and filter the stream based on key(like type1, type2, error etc.) and again create multiple streams. But none of them seems to me a good design. What would be the best way to achieve it?

SQL.injection
  • 2,607
  • 5
  • 20
  • 37
dreamer
  • 1,039
  • 2
  • 16
  • 36

1 Answers1

0

AFAIK, you can't do that. To split a stream, you will need to do the following:

// main stream
Stream stream = topology.each(...)

// stream 01
Stream stream1 = stream.each(...)

// stream 02
Stream stream2 = stream.each(...)
Luiz Picanço
  • 359
  • 4
  • 9