Oozie example stuck when run pig job

Question

I start test oozie job on cluster with yarn "oozie job -config /usr/lib/oozie/oozie-4.0.1/examples/apps/pig/job.properties -run" and my job stuck on 0% and then send only heart beat

2014-06-25 12:51:57,800 [JobControl] INFO  org.apache.hadoop.yarn.client.RMProxy  - Connecting to ResourceManager at localhost/127.0.0.1:8032
2014-06-25 12:51:57,831 [JobControl] INFO  org.apache.hadoop.conf.Configuration.deprecation  - fs.default.name is deprecated. Instead, use fs.defaultFS
2014-06-25 12:51:58,021 [JobControl] INFO  org.apache.hadoop.mapreduce.lib.input.FileInputFormat  - Total input paths to process : 1
2014-06-25 12:51:58,021 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths to process : 1
2014-06-25 12:51:58,021 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths to process : 1
2014-06-25 12:51:58,022 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths (combined) to process : 1
2014-06-25 12:51:58,022 [JobControl] INFO  org.apache.pig.backend.hadoop.executionengine.util.MapRedUtil  - Total input paths (combined) to process : 1
2014-06-25 12:51:58,034 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - number of splits:1
2014-06-25 12:51:58,084 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Submitting tokens for job: job_1403700612904_0002
2014-06-25 12:51:58,084 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: mapreduce.job, Service: job_1403700612904_0001, Ident: (org.apache.hadoop.mapreduce.security.token.JobTokenIdentifier@63ad6884)
2014-06-25 12:51:58,085 [JobControl] INFO  org.apache.hadoop.mapreduce.JobSubmitter  - Kind: RM_DELEGATION_TOKEN, Service: 127.0.0.1:8032, Ident: (owner=test, renewer=oozie mr token, realUser=root, issueDate=1403700700415, maxDate=14043$
2014-06-25 12:51:58,352 [JobControl] INFO  org.apache.hadoop.yarn.client.api.impl.YarnClientImpl  - Submitted application application_1403700612904_0002 to ResourceManager at localhost/127.0.0.1:8032
2014-06-25 12:51:58,393 [JobControl] INFO  org.apache.hadoop.mapreduce.Job  - The url to track the job: http://localhost:8088/proxy/application_1403700612904_0002/
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - HadoopJobId: job_1403700612904_0002
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - HadoopJobId: job_1403700612904_0002
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - Processing aliases A,B
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - Processing aliases A,B
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - detailed locations: M: A[18,4],B[19,4] C:  R:
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - detailed locations: M: A[18,4],B[19,4] C:  R:
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1403700612904_0002
2014-06-25 12:51:58,394 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - More information at: http://localhost:50030/jobdetails.jsp?jobid=job_1403700612904_0002
2014-06-25 12:51:58,429 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
2014-06-25 12:51:58,429 [main] INFO  org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher  - 0% complete
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat
Heart beat

can you share your workflow.xml and pigscript – K S Nidhin Jun 25 '14 at 16:03 — K S Nidhin, Jun 25 '14 at 16:03

score 0 · Accepted Answer · edited Jul 13 '14 at 05:49

0

Me too faced same problem while scheduling pig-0.12.1 in oozie-4.0.1 with hadoop-2.2.0.

I cant able to schedule pig script using oozie in hadoop-2.2.0 in single node cluster. But I did it in multinode cluster by doing the following changes.

NodeManager and Resource Manager running in same system. So i am gettting this error. I started Node manager in second system and my problem is solved.

edited Jul 13 '14 at 05:49

elixenide

44,308
16
74
100

answered Jul 04 '14 at 11:01

Suresh Ram

1,034
3
16
40

If I run more than two oozie job concurrently mean it get fail to run and same problem raise again. Because when run oozie job mean launcher job was creates, it occupy one nodemanager. This solution was not help in all case. – ǨÅVËĔŊ RĀǞĴĄŅ Dec 31 '14 at 10:10

score 0 · Answer 2 · edited May 23 '17 at 12:12

0

Hi Heart beat error is occurs due to insufficient Total memory size in Hadoop. This because you may run in small cluster machine.

Solution: So you want to increase the size of Total memory to all nodemanager to perform mapreduce job. Steps are followed in this link "HEART BEAT ERROR SOLUTION"

edited May 23 '17 at 12:12

Community

1
1

answered Jan 09 '15 at 09:09

ǨÅVËĔŊ RĀǞĴĄŅ

530
6
30

Oozie example stuck when run pig job

2 Answers2