1

I am running hive on a standalone machine. Hadoop is running in pseudo-distributed mode. I am running hive query which joins two tables (one table has 7M and another has 51M records and each containing 8 columns). After processing some time, Mapper reaches to zero percentage and then continues to print zero occasionally. Can you please help me to resolve this.

For reference find the logs below.

2016-04-12 22:52:58,469 Stage-1 map = 71%,  reduce = 1%
2016-04-12 22:53:00,517 Stage-1 map = 72%,  reduce = 1%
2016-04-12 22:53:02,560 Stage-1 map = 73%,  reduce = 1%
2016-04-12 22:53:09,740 Stage-1 map = 74%,  reduce = 1%
2016-04-12 22:53:11,796 Stage-1 map = 75%,  reduce = 1%
2016-04-12 22:53:13,842 Stage-1 map = 76%,  reduce = 1%
2016-04-12 22:53:21,037 Stage-1 map = 77%,  reduce = 1%
2016-04-12 22:53:24,114 Stage-1 map = 78%,  reduce = 1%
2016-04-12 22:53:26,156 Stage-1 map = 79%,  reduce = 1%
2016-04-12 22:53:35,433 Stage-1 map = 81%,  reduce = 1%
2016-04-12 22:53:38,507 Stage-1 map = 82%,  reduce = 1%
2016-04-12 22:53:45,725 Stage-1 map = 82%,  reduce = 0%
2016-04-12 22:53:49,925 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:54:50,236 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:55:50,546 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:56:50,863 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:57:51,128 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:58:51,352 Stage-1 map = 0%,  reduce = 0%
2016-04-12 22:59:51,612 Stage-1 map = 0%,  reduce = 0%
2016-04-12 23:00:51,886 Stage-1 map = 0%,  reduce = 0%
2016-04-12 23:01:52,131 Stage-1 map = 0%,  reduce = 0%

I verified the status in the tracker. The status was showing two attemps and one attempt was failed with the diagnostic message as follows.

AM Container for appattempt_1460481465127_0001_000001 exited with exitCode: -100
For more detailed output, check application tracking page:http://localhost:8088/cluster/app/application_1460481465127_0001Then, click on links to logs of each attempt.
Diagnostics: Container released on a *lost* nodeFailing this attempt

Thanks in advance.

Santhosh Tangudu
  • 759
  • 9
  • 19

1 Answers1

0

The problem seems to be caused by Heap space on the map side.
Try to increase the map task heap size by doing the following:

in mapred-site.xml (Try to tune the following values to match with your use case):

<!-- for mappers -->
<property>
   <name>mapreduce.map.memory.mb</name>
   <value>4096</value>
</property>
<!-- For reduces -->
<property>
   <name>mapreduce.reduce.memory.mb</name>
   <value>8192</value>
</property>
user1314742
  • 2,865
  • 3
  • 28
  • 34