0

I'm just starting with storm. I have a simple topology of a spout (1 executor) and a bolt (4 executors). The spout and the bolt are connected via "shuffleGrouping".

From what I can see the spout sends tuples only to a subset of bolt executors - the ones that are running on the same host as the spout.

Is this expected? Is there a way to spread the load across all bolt executors no matter where they run?

facha
  • 11,862
  • 14
  • 59
  • 82

2 Answers2

0

Yes, it's expected. I think you can config it by using same spout source, like storm integrate with kafka,then you create 2 topologies, config each spout of each topology with the same topic and same zookeeper host.

Pham Hung
  • 43
  • 4
0

Sorry for digging up this old post. I had the same problem with a topology and found this post, so I want to share my research, and a/the solution.

I upgraded a topology from Apache Storm 1.0.2 to 2.2.0. Since, the spout sends tuples only to the local bolt executors despite the SHUFFLE grouping mode.

According to this post, it's seems expected, but there are a workaround :

topology.disable.loadaware.messaging: true

See also org.apache.storm.daemon.GrouperFactory to understand how this configuration is used.

Mikwiss
  • 1