1

I was trying to study apache storm and setup the storm on my ubuntu installed in vmware. storm version is 2.4.0 and it is running on localhost currently. When I submit the topology. Topology is submitted successfully and distributed according to default scheduler but bolts and spouts never emit any data. The value of emitted column in topology details remains 0. In Owner's Summary it gives following error: "This user's topologies are in danger of being unscheduled due to the owner's over-use of cluster resources". Similar question is already asked here but it is also not answered.

storm.yaml

enter image description here

owner's summary screen

enter image description here

I tried many things like changing yaml file trying setup on different machines but it gets the same error.

James Z
  • 12,209
  • 10
  • 24
  • 44
Amjad Arshad
  • 11
  • 2

1 Answers1

0

То solve this red notice you should add these config lines:

  • to storm.conf
# available memory for your supervisor
supervisor.memory.capacity.mb: 8192.0
# available CPU resources for supervisor, 100 per core
supervisor.cpu.capacity: 1200.0
# scheduler class
storm.scheduler: "org.apache.storm.scheduler.resource.ResourceAwareScheduler"
  • create file user-resource-pools.yaml in your storm conf dir. Content: ubuntu - username you topologies run from, exampleuser - just for syntax example
resource.aware.scheduler.user.pools:
    ubuntu:
        cpu: 1200.0
        memory: 8192.0
    exampleuser:
        cpu: 100.0
        memory: 1024.0
  • restart your nimbus and supervisor

More about ResourceAwareScheduler you can find in storm 2.4.0 docs and storm 1.2.3 docs

By the way, check your worker.log at logs/worker-artifacts/your-topology-id/.

If emited tuples are 0 some bolts or spouts are dead.

Vitos
  • 121
  • 6
  • Can you explain, why there is an error related to the RAS when the RAS was not even activated according to the original `storm.yaml`? – moosehead42 Jan 11 '23 at 07:29
  • @moosehead42 The owner.html [link](https://github.com/apache/storm/blob/f481eaf043dba3b6c62ac27bb2306ef652f2ace9/storm-webapp/src/main/java/org/apache/storm/daemon/ui/WEB-INF/owner.html) report uses /api/v1/owner-resources/?owner-name that returns resources in use. In the case of the default scheduler, it returns "cpuGuarantee":"N/A" instead of "cpuGuarantee":1200.0 in the case of RAS, which causes displaying that red banner. – Vitos Jan 11 '23 at 08:35