How do you get jobs & tasks statistics in Azure batch with the Python API?

Question

I started working with Azure batch recently and found I could not get statistics for a given job although fields for that exists.

I am using the Python API, when I poll for job information the stats field is always None. The JobListOptions class can be used with the job.list method but documentation is not very useful.

Going through the BatchExplorer sample code I found out that the expand field of the JobListOptions class might require to be set to stats but I'm still unsure as I'm still not getting stats from the Python API.

fpark · Answer 1 · 2018-01-11T18:48:15.727

3

You can get stats from jobs as follows:

my_batch_client.job.list(
    job_list_options=azure.batch.models.JobListOptions(
        expand='stats'
    )
)

Note that it's more efficient to get a job rather than list all of your jobs if you're just interested in a single job. In that case use my_batch_client.job.get() with the appropriate JobGetOptions.

Or for all tasks under a job:

my_batch_client.task.list(
    job_id='MY_JOB_ID',
    task_list_options=azure.batch.models.TaskListOptions(
        expand='stats'
    )
)

Similarly, if you are only interested in a single task under a job, query just that task as it's more efficient:

my_batch_client.task.get(
    job_id='MY_JOB_ID',
    task_id='MY_TASK_ID',
    task_get_options=azure.batch.models.TaskGetOptions(
        expand='stats'
    )
)

edited Jan 11 '18 at 18:48

answered Jul 10 '17 at 15:05

fpark

2,304
2
14
21

I did actually found the answer on my own, but you did write the answer first! – Rastikan Jul 10 '17 at 19:05
It would be a good answer if you could also provide an example to get the status of individual tasks. – void Jan 10 '18 at 11:59
Updated the answer for a single task query. – fpark Jan 11 '18 at 18:49

score 0 · Answer 2 · answered Jul 10 '17 at 09:07

It sounds like you want to get the resource usage statistics for the entire lifetime of jobs & tasks of Azure Batch using Azure Python SDK, so first you need to turn on diagnostics to collect the ServiceLog logs as the figure below.

Then you can get these statistics stats from the response of the methods batch_client.job.get(JOB_ID) & batch_client.task.get(JOB_ID, TASK_ID).

How do you get jobs & tasks statistics in Azure batch with the Python API?

2 Answers2