Stack Overflow
Stack Overflow

Questions tagged [azure-batch]

The azure-batch tag should be used to ask questions regarding all aspects of the Azure Batch service including job scheduling, compute node provisioning, and pool orchestration. Extended ecosystem questions are also welcome that are related to Azure Batch accelerator offerings, Azure Portal Batch blades, and Batch Explorer.

Azure Batch is a job scheduling and compute management orchestration solution for the Microsoft Azure cloud. The service is available in all Azure regions and can deploy with nearly all Azure VM sizes available, including those with specialized hardware such as GPUs and Infiniband/RDMA interfaces.

Azure Batch is a managed job scheduling solution where the underlying compute resources are allocated as pools. Pools may be comprised of Linux or Windows VMs and any number of dedicated and low priority nodes and can be automatically scaled through autoscale formulas. An Azure Batch Job can be thought of as a queue for tasks. Each Azure Batch Job can be assigned to a pool. Any number of tasks can be assigned to jobs, including multi-instance tasks (i.e., multi-node) which support MPI jobs. Azure Batch also supports containerized execution.

For a full introductory treatment of Azure Batch, see the Azure Batch Technical Overview. The service team also provides a GitHub repository for issue tracking, feature requests, and change logs.

416 questions
0
votes
1 answer

Setting up a Job Schedule

I currently have a setup that creates a job and then collect some metrics about the tasks in the job. I want to do something similar, but by setting a job schedule instead. In particular, I want to set a job schedule that wakes up at a recurrence…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

Running Parallel Tasks in Batch

I have few questions about running tasks in parallel in Azure Batch. Per the official documentation, "Azure Batch allows you to set maximum tasks per node up to four times (4x) the number of node cores." Is there a setup other than specifying the…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

BlobAccessDenied with Application packages

I have been having an issue with application packages where there will be days/times where I just get a null exit code and BlobAccessDenied Error for the exact same application package and exact same command line that previously worked and gave a…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

Microsoft Azure Batch Service Account Create from .NET

I was looking into creating a Microsoft Azure Batch Account from the .NET SDK. I was successful in authentication but I just came across this error: Microsoft.Rest.Azure.CloudException: The client 'xxxxxxxxxxxxxxxxxxxxxxxxxx' with object id …
UmairAhmad
  • 150
  • 3
  • 14
0
votes
0 answers

Azure Batch Pool

I am creating a custom activity pipeline in Azure Data Factory and as a part of that I need to create an AzureBatchLinkedService using the JSON below: { "name": "AzureBatchLinkedService", "properties": { "hubName": "name_hub", …
SUMguy
  • 1,505
  • 4
  • 31
  • 61
0
votes
1 answer

Retrieving the number of tasks in a particular state in the Azure Batch API

The Azure Batch API provides the list function, which retrieves an enumerable list of tasks in a job, which takes TaskListOptions, to, for instance, filter the tasks by state. I would like to query the API only for the number of tasks in a…
Christoph Wintersteiger
  • 8,234
  • 1
  • 16
  • 30
0
votes
1 answer

Batch nodes restarting and batch pool settings

I have couple questions about Azure Batch pool: I noticed that sometimes, while running a job, especially when I'm running a large number of tasks like 10000, some/many compute nodes in my pools shut down by themselves and restart. I would like to…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

BadRequest when adding expandClause for JobStatistics

I want to get some statistics about the job I'm running on my pool, and for that I am trying to use the JobStatistics class, but I have been getting job.Statistics as null in most of my runs except for few where the result was magically not null. I…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

Batch Environment Variables returning null

I am trying to use the Batch service-defined environment variables, but I keep getting a null error. Similar to what I saw in the samples, I am trying to get the jobID for a job that I created through Azure portal as follows: private readonly string…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

Packaging Applications for Azure Batch

I am having trouble packaging applications to get them to run in Azure Batch compute nodes. I am using user subscription with VM configuration, so I can't use application packages. I have been uploading my executable files and dlls as resource…
J.B
  • 137
  • 2
  • 16
0
votes
2 answers

How do you get jobs & tasks statistics in Azure batch with the Python API?

I started working with Azure batch recently and found I could not get statistics for a given job although fields for that exists. I am using the Python API, when I poll for job information the stats field is always None. The JobListOptions class can…
Rastikan
  • 517
  • 5
  • 18
0
votes
1 answer

Task and Job Statisitics - WallClockTime and ExecutionInformation

I am trying to collect some statistics and performance metrics about the cloud tasks and jobs that I am submitting to the Azure Batch pool. For that, I am using the built-in TaskExecutionInformation and TaskStatisitics classes, but I am confused…
J.B
  • 137
  • 2
  • 16
0
votes
1 answer

Windows command: Set exit code of chained commands based on first command

i have this case scenario (this is command that is executed on Azure Batch task): cmd /c MainApp.exe & Util.exe What i need is to exit this whole command with exit code from MainApp.exe as Util.exe exits always with 0 code. I have tried something…
Daniel
  • 149
  • 1
  • 1
  • 8
0
votes
1 answer

Piping output in cmd batch file on Azure Batch service error codes not detected

I have a .cmd batch file (let's call it RunSQlCmd.cmd) which pipes out output of sqlcmd to 7 zip compressor sqlcmd -i.\table.sql -S . -E -s "," -I -h -1 -W| "c:\Program Files\7-Zip\7z.exe" a -tbzip2 -si "out.csv.bz2" I run it from Azure Batch in a…
YuGagarin
  • 341
  • 7
  • 20
0
votes
1 answer

Issue with Azure Batch Job, ending with unkown error code

I'm having problems with Azure Batch Jobs. I'm trying to create an app pool, create a CloudTask and then execute my Application Package being online. Do you see something not working correctly? Here is the code used now. Main code: await…
user2687506
  • 789
  • 6
  • 21
1 2 3
…
27 28