Stack Overflow
Stack Overflow

Questions tagged [azure-batch]

The azure-batch tag should be used to ask questions regarding all aspects of the Azure Batch service including job scheduling, compute node provisioning, and pool orchestration. Extended ecosystem questions are also welcome that are related to Azure Batch accelerator offerings, Azure Portal Batch blades, and Batch Explorer.

Azure Batch is a job scheduling and compute management orchestration solution for the Microsoft Azure cloud. The service is available in all Azure regions and can deploy with nearly all Azure VM sizes available, including those with specialized hardware such as GPUs and Infiniband/RDMA interfaces.

Azure Batch is a managed job scheduling solution where the underlying compute resources are allocated as pools. Pools may be comprised of Linux or Windows VMs and any number of dedicated and low priority nodes and can be automatically scaled through autoscale formulas. An Azure Batch Job can be thought of as a queue for tasks. Each Azure Batch Job can be assigned to a pool. Any number of tasks can be assigned to jobs, including multi-instance tasks (i.e., multi-node) which support MPI jobs. Azure Batch also supports containerized execution.

For a full introductory treatment of Azure Batch, see the Azure Batch Technical Overview. The service team also provides a GitHub repository for issue tracking, feature requests, and change logs.

416 questions
2
votes
0 answers

Azure batch pool does not allocate nodes

I have created an azure batch account using "user subscription" allocation mode in order to control the network where my nodes will belong. The objective is to be able to open some firewalls for the set of IP that the nodes may take. I had been…
Bruno Belmondo
  • 2,299
  • 8
  • 18
2
votes
1 answer

Is deleting tasks and pools necessary in Azure batch jobs?

The Azure Batch .net tutorial shows deleting tasks, then jobs, then pools. At the same time, I found this article about cleanup, instead of deleting jobs and pools, it just deletes VMs which are no longer used. Is this a good strategy? I see that it…
Nomenator
  • 1,021
  • 1
  • 13
  • 28
2
votes
1 answer

Azure batch insert - how to reduce response length?

I use Azure client lib to perform batch inserts into Azure Table Storage. Everything works fine. But when I sniff requests using Fiddler I found that every response from azure is about 90KB. I've changed prefer header to "return-no-content", but the…
pyskaty
  • 125
  • 1
  • 4
1
vote
2 answers

Azure Batch pool quota reached

when creating a pool for my Batch account I get the following error message: Pool quota for the account has been reached I currently have no other pool created and when checking the quota of the batch account in the portal, the quota for pools is…
jasdefer
  • 757
  • 12
  • 24
1
vote
1 answer

When I Create a CloudPool used SubnetId, I got a "BadRequest"

I want to create a cloudpool with vnet,I typed codes: 1.use AzureAD Task tokenProvider() => GetAuthenticationTokenAsync(); using BatchClient batchClient = BatchClient.Open(new BatchTokenCredentials(_param.BatchAccountUrl,…
Zhouping Wang
  • 39
  • 2
1
vote
1 answer

Access Azure Key Vault with Managed Identity on Azure Batch running .NET Console Application

Is it possible to access Azure Key Vault Secrets with Azure Managed Identity setup on the Azure Batch? I've read through Microsoft docs and the only option I technically see is using certificate based authentication to access key vault within the…
A. Wiley
  • 11
  • 3
1
vote
1 answer

How to trigger/force Azure Batch pool autoscale formula

I created a pool in Azure Batch (from the Azure portal) with Auto scale activated. I also defined a formula where the initial number of node is set to 0. This number will ramp up according to number of active tasks and will go back to 0 if there no…
wad_
  • 11
  • 1
1
vote
0 answers

Unable to Create Pool using Azure Batch Management Library with ServiceClientCrenetials generated via AzureCredentialsFactory

I am using this documentation to assign a managed identity to my Batch Pool. For simplicity I do not include that assignment in the examples below as the issue is not tied to that but rather to accessing the management library with my credentials…
INNVTV
  • 3,155
  • 7
  • 37
  • 71
1
vote
1 answer

How do i set up my job schedule in Azure batch account with different timedelta?

I want my job can run at the first day of each mouth,but in azure batch account i only set up a fixed timedelta(Recurrence interval),how can i set up?
Frank.H
  • 13
  • 2
1
vote
1 answer

Azure Batch won't auto-scale down

I've created a pool and a job tied to a pool in Azure Batch. After I add a task to the job, I can see that the active tasks from this auto-scale formula returns an array of '1's after a few minutes: $tasks =…
skunkwerk
  • 2,920
  • 2
  • 37
  • 55
1
vote
1 answer

Azure Batch Needs to be Updated to PowerShell Version 7+

I am running a Custom Activity in Azure Data Factory. We are using the Test-Json command to validate JSON data against a schema. This command works perfectly find in PowerShell version 7+ but will fail in version 5.1. For testing purposes, if I…
Bill Donofrio
  • 21
  • 2
1
vote
1 answer

Create Azure Batch activity in Data factory

I want to create an Azure Batch Activity in my Data Factory Pipe, I set up a trigger that checks for new "last modified" blobs in the last 24 hrs. As I'm dealing with big files I want to leverage the power of Azure Batch and multiprocess 2 blobs at…
3nomis
  • 1,175
  • 1
  • 9
  • 30
1
vote
1 answer

Azure ADF using Azure Batch throws Shared Access Signature generation error

I am working on a simple Azure Data Factory pipeline where I have simply added a Batch Service and in that specified the Batch Service account (which I have created thru linked service and tested the connection is working). In the command I am just…
user42012
  • 722
  • 12
  • 33
1
vote
1 answer

Install R on the nodes for Azure batch services

I can create the batch service resources using Power shell as described here: https://learn.microsoft.com/en-us/azure/batch/batch-powershell-cmdlets-get-started I want to run a R script on the nodes and I need R installed on the nodes as none of the…
Sindu_
  • 1,347
  • 8
  • 27
  • 67
1
vote
1 answer

Set Azure Batch MaxWallClockTime Node SDK

I'm trying to set a maxWallClockTime of 72 hours using the ISO 8601 Duration format. The documentation for this property is useless, so I'm basing my guess on using the 8601 format on that being the way to set the same property at the Batch Job…
Mark
  • 816
  • 1
  • 9
  • 27
1 2 3
…
27 28