0

I modified the RAM requirement in the Heron example topology named WordCountTopology.java and rebuild the file using mvn assembly:assembly command. When I submitted the modified WordCountTopology to Heron cluster, I found the RAM requirement of Heron Instance did not changed.
The process of building .jar is succeed. The default RAM requirement of the WordCountTopology as following:

// configure component resources
conf.setComponentRam("word",
        ByteAmount.fromMegabytes(ExampleResources.COMPONENT_RAM_MB * 2));
conf.setComponentRam("consumer",
        ByteAmount.fromMegabytes(ExampleResources.COMPONENT_RAM_MB * 2));

// configure container resources
conf.setContainerDiskRequested(
    ExampleResources.getContainerDisk(2 * parallelism, parallelism));
conf.setContainerRamRequested(
    ExampleResources.getContainerRam(2 * parallelism, parallelism));
conf.setContainerCpuRequested(2);

In the above code. ExampleResources.COMPONENT_RAM_MB = 512mb. The default value of parallelism is 1.
The content about ExampleResources as following:

static ByteAmount getContainerDisk(int components, int containers) {
    return ByteAmount.fromGigabytes(Math.max(components / containers, 1));
  }

  static ByteAmount getContainerRam(int components, int containers) {
    final int componentsPerContainer = Math.max(components / containers, 1);
    return ByteAmount.fromMegabytes(COMPONENT_RAM_MB * componentsPerContainer);
  }

My changed the value of ExampleResources.COMPONENT_RAM_MB=512mb to 256mb.
However, the requirement of the topology showed in the Aurora scheduler as following: enter image description here And all instances in the aurora is FAILED: enter image description here My Questions: What should I do to effectively change the RAM requirement in the topology？And I don't know why tasks failed running in the mesos and aurora. Thanks for your help.

Jonas
  • 121,568
  • 97
  • 310
  • 388
Yitian Zhang
  • 314
  • 1
  • 3
  • 18

1 Answers1

0

Do you know which version of Heron are you using? We recently cut over to a new packing algorithm called Resource Compliant Round Robin scheduling. Eventually, the resource allocation will be automatic.

Karthik
  • 132
  • 2
  • I made a mistake. I know what the problem is now. What is displayed in Aurora Scheduler is the allocated memory resources, not the amount of memory used by the task running. So the memory requirements for the components here have actually been modified. By the way, I used the version of heron is 0.17.1. Could you give me your resources link of Resource Compliant Round Robin scheduling in Github or somewhere? – Yitian Zhang Apr 13 '18 at 03:19