1

From the URL https://developer.ibm.com/hadoop/2017/06/30/deep-dive-yarn-cgroups/ we see the following CPU limit formulas for yarn cgroups ,my question is those formulas still valid if there are many threads (suppose 500 threads and each doing the CPU intensive operation such as infinite loop operation) running in each container? That's to say if the yarn cgroups still can control CPU usage if the number of threads far more that the number of vcores of each container?

"In hard limit mode, YARN uses the following formula to calculate the CPU limit for each container.

C: number of physical CPU cores
P: strict limit percent
V: number of vcores on a single node manager
A: number of vcores for a single container

CPU_LIMIT = C * P * A / V;
In our example, a single container requests one vcore. The CPU limit for this container will be

CPU_LIMIT = C * P * A / V = 8 * 50% * 1 / 6 = 2/3 = 66.7%

And if there is another container requesting 2 vcores, the CPU limit for that container will be

CPU_LIMIT = C * P * A / V = 8 * 50% * 2 / 6 = 4/3 = 133%"
YuFeng Shen
  • 1,475
  • 1
  • 17
  • 41

0 Answers0