I am currently running two jobs such that job2 is dependent on job1. I plan to use output of job1 and pass it as distributed cache in job2. This will be used as reference in job2 to process another set of data. I have written load functions in the 2 jobs where i have done job configurations. Then i make job2 dependent on job1 and submit both it to JobControl. The problem is when i use distributed cache inside my load function in job2, the file is not yet created, and i will get a Exception. How do i handle this?
I couldn't see any initialize() etc method which i can overload instead of using my load() functions. I am using the new apis.
MyTranslationBuilderDriver job1 = new MyTranslationBuilderDriver();
job1.load(args, "translator/path");
MyTranslatorDriver job2 = new MyTranslatorDriver();
job2.load(args, "path");
JobControl jbcntrl=new JobControl("jbcntrl");
ControlledJob cjob1 = new ControlledJob(job1.job, null);
ControlledJob cjob2 = new ControlledJob(job2.job, null);
cjob2.addDependingJob(cjob1);
jbcntrl.addJob(cjob2);
jbcntrl.run();
Thread thread = new Thread(jbcntrl);
thread.start();
while (!jbcntrl.allFinished()) {
try {
Thread.sleep(10000);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
jbcntrl.stop();
}