Resource Manager: How to get list of app ids that were executed in a given job queue

Question

I want to get a list of YARN app ids that were executed in a MapReduce job queue.

I explored Resource Manager API but could not find an API which simply requires me to pass job queue name and get the list of app ids.

Is there a way to achieve this?

Answer 1

The Resource Manager Cluster Applications API accepts queue as parameter for filtering. However, this API will only show you current jobs. For jobs that have completed you need to look at the Job History server. The Job History Server's Jobs API shows completed jobs and also supports queue as a parameter for filtering.

curl JOB_HIST_URL/ws/v1/history/mapreduce/jobs?queue=QUEUENAME

https://hadoop.apache.org/docs/stable/hadoop-mapreduce-client/hadoop-mapreduce-client-hs/HistoryServerRest.html#Jobs_API