2

I am running a hive query and I got the following error when setting the hive.execution.engine=tez, while the query is working under engine=MR. 

FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.tez.TezTask

My query is an inner join and the data is quite big.

Another thing is that I have met this problem before. But tez works later so I thought it was about something unstable about hive.

Vicky
  • 1,465
  • 2
  • 12
  • 21
  • I thought your question title was meant to be "Have this execution error". Was going to edit it , then realized the tags :D – m4n0 Nov 05 '15 at 22:07
  • The title is meant to be Hive execution error with setting execution engine as tez. – Vicky Nov 05 '15 at 22:11

3 Answers3

2

While running your HQL via hive include following parameter. This will give you detailed logs and you can determine the root cause easily.

-hiveconf hive.root.logger=DEBUG,console

I faced similar problem and above property help me big time.

e.g.: I got following message

16/04/14 10:29:26 ERROR exec.Task: Failed to execute tez graph.
org.apache.tez.dag.api.TezException: org.apache.hadoop.yarn.exceptions.InvalidResourceRequestException: Invalid resource request, requested memory < 0, or requested memory > max configured, requestedMemory=20480, maxMemory=11288

When I changed my setting to 11288, my query went through fine.

halfer
  • 19,824
  • 17
  • 99
  • 186
Ambrish
  • 3,627
  • 2
  • 27
  • 42
0

Once check your yarn-site.xml with following properties.

<property>
   <name>yarn.nodemanager.vmem-check-enabled</name>
    <value>false</value>
    <description>Whether virtual memory limits will be enforced for containers</description>
  </property>
 <property>
   <name>yarn.nodemanager.vmem-pmem-ratio</name>
    <value>4</value>
    <description>Ratio between virtual memory to physical memory when setting memory limits for containers</description>
  </property>
 <property>
    <name>yarn.scheduler.minimum-allocation-mb</name>
    <value>1024</value>
  </property>
  <property>
    <name>yarn.scheduler.maximum-allocation-mb</name>
    <value>2048</value>
  </property>
  <property>
    <name>yarn.nodemanager.resource.memory-mb</name>
    <value>2048</value>
  </property>
</configuration>
ROOT
  • 1,757
  • 4
  • 34
  • 60
0

Found this post, which made it work for me. Needed to add the username

hadoop

VadimBelov
  • 671
  • 6
  • 3