I have the following set in my mapred-site.xml in my ec2 inctance where my Hadoop is working :
<property>
<name>mapreduce.map.memory.mb</name>
<value>2048</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>4096</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx1024m</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx3072m</value>
</property>
With the above properties set Sqoop works perfectly but Hive Tez doesnot. So I changed it to the following:
<property>
<name>mapreduce.map.memory.mb</name>
<value>4096</value>
</property>
<property>
<name>mapreduce.reduce.memory.mb</name>
<value>8192</value>
</property>
<property>
<name>mapreduce.map.java.opts</name>
<value>-Xmx3072m</value>
</property>
<property>
<name>mapreduce.reduce.java.opts</name>
<value>-Xmx6144m</value>
</property>
as Hive Tez needs more memory, Sqoop stops working and hive Tez gives me error like:
Diagnostic Messages for this Task:
Container [pid=51661,containerID=container_e50_1493005386967_25486_01_000243] is running beyond virtual memory limits. Current usage: 274 MB of 1 GB physical memory used; 2.7 GB of 2.1 GB virtual memory used. Killing container
Suggest me how can I make both Sqoop and Hive tez work.