Hive Container is running beyond physical limits

Question

I frequently get the error message in hive container running beyond physical memory limits. Current usage: 4.7 GB of 4GB physical memory used; 12.1 GB of 8.4 GB virtual memory used. Killing container

I execute this in scala using jdbc connection.

My default execution engine is tez. If change hive execution engine to mr it is working fine.

If I execute the same query in hive cli, on tez it works fine.

As said in various forum I tried to change the container size, but it doesnt help.

SET hive.tez.container.size=20000
SET hive.tez.java.opts=-Xmx10240m -XX:NewRatio=8

It seems above settings doesnt effectively increase the container size.

Also it is hogging the whole cluster.

My question is:

1. Is there any way which we can restrict number of containers spawned for a hive query, so that cluster doesnt halt because of the big queries?
2. How can we increase the size of container?

3. Is there any way out other than increasing the container size?

   hive version: 1.2.1

   hdp: 2.3.2

Answer 1

it's caused by YARN's limitation on task memory usage inside a container.

yarn.nodemanager.pmem-check-enabled  true   Whether physical memory limits will be enforced for containers.
yarn.nodemanager.vmem-check-enabled  true   Whether virtual memory limits will be enforced for containers.

You could disable the pmem and vmem check in yarn-site in your yarn configuration folder.

Thanks

Answer 2

hive> set mapreduce.map.memory.mb = 9000;
hive> set mapreduce.map.java.opts =- Xmx7200m;
hive> set mapreduce.reduce.memory.mb = 9000;
hive> set mapreduce.reduce.java.opts =- Xmx7200m;