parallel execution of mapreduce jobs

Question

I have a question. can I execute two or more jobs in hadoop concurrently with JobCntrol? I have 2 jobs that can be execute in parallel and other jobs have dependencies with these two jobs? How can I do that with JobControl? when I use it, it gives the error: java.lang.IllegalStateException: Job in state RUNNING instead of DEFINE

score 1 · Accepted Answer · answered Oct 10 '13 at 10:53

1

There is no need for a JobControl when you want to execute the jobs in parallel. Just submit the jobs using JobClient.runJob() API. JobControl is used when there is a simple DAG of jobs.

answered Oct 10 '13 at 10:53

Praveen Sripati

32,799
16
80
117

thanks a lot for reply.. and another question.. I want to know what is chaining jobs for? I mean chainMapper and chainReducer. – Anse danesh Oct 11 '13 at 09:00
@ Anse danesh : Inorder to know about chaining jobs , I have written a blog on it : http://www.unmeshasreeveni.blogspot.in/2014/04/chaining-jobs-in-hadoop-mapreduce.html – USB Jun 12 '14 at 04:35

parallel execution of mapreduce jobs

1 Answers1