What is the difference between oozie and Hadoop schedulers (FIFO, Fair, Capacity, Delay etc) ? Which one is normally used in production environments ?
Asked
Active
Viewed 234 times
1 Answers
1
They offer different functions.
Oozie is a job scheduler and orchestration engine. It'll execute YARN jobs within the configured queue. For example, mapreduce.job.queuename in MR/Java action, tez.queue.name in Hive Action (on Tez), or --queue in Spark.
The queue you use depends on the workloads you have, but Fair or Capacity work best for multi-tenant workloads.
FIFO ordering within a queue works best if you have jobs that need to run before others, although Oozie has its own definition of how to order a full job
OneCricketeer
- 179,855
- 19
- 132
- 245