Im trying to run a job in hive with cluster(1 master, 4 core nodes[11.25GB each]) in AWS EMR, im joining(MAP joining) two tables one with 0.3 million entries(~11mb) and another table with almost 7 million entries(took care that big table should be last). Since, it is map join, first small table is getting hashed in distributed cache, after hashing some entries iam getting this error.
2015-10-07 11:21:35 Processing rows: 10600000 Hashtable size: 10599999 Memory usage: 5465925664 percentage: 0.143
2015-10-07 11:22:09 Processing rows: 10700000 Hashtable size: 10699999 Memory usage: 6078125280 percentage: 0.159
2015-10-07 11:22:42 Processing rows: 10800000 Hashtable size: 10799999 Memory usage: 2698310976 percentage: 0.071
2015-10-07 11:23:16 Processing rows: 10900000 Hashtable size: 10899999 Memory usage: 3272893216 percentage: 0.086
2015-10-07 11:23:49 Processing rows: 11000000 Hashtable size: 10999999 Memory usage: 3821330720 percentage: 0.10
2015-10-07 11:24:23 Processing rows: 11100000 Hashtable size: 11099999 Memory usage: 4398775280 percentage: 0.115
2015-10-07 11:24:57 Processing rows: 11200000 Hashtable size: 11199999 Memory usage: 5020884784 percentage: 0.132
2015-10-07 11:25:30 Processing rows: 11300000 Hashtable size: 11299999 Memory usage: 5608609488 percentage: 0.147
2015-10-07 11:26:04 Processing rows: 11400000 Hashtable size: 11399999 Memory usage: 6146767072 percentage: 0.161
2015-10-07 11:26:38 Processing rows: 11500000 Hashtable size: 11499999 Memory usage: 2622476744 percentage: 0.069
2015-10-07 11:27:11 Processing rows: 11600000 Hashtable size: 11599999 Memory usage: 3224517304 percentage: 0.084
2015-10-07 11:27:44 Processing rows: 11700000 Hashtable size: 11699999 Memory usage: 3793226384 percentage: 0.099
Execution failed with exit status: 137
Obtaining error information
Task failed!
Task ID:
Stage-4
Logs:
/mnt/var/log/apps/hive.log
FAILED: Execution Error, return code 137 from org.apache.hadoop.hive.ql.exec.mr.MapredLocalTask
