Stack Overflow
Stack Overflow

Questions tagged [azure-batch]

The azure-batch tag should be used to ask questions regarding all aspects of the Azure Batch service including job scheduling, compute node provisioning, and pool orchestration. Extended ecosystem questions are also welcome that are related to Azure Batch accelerator offerings, Azure Portal Batch blades, and Batch Explorer.

Azure Batch is a job scheduling and compute management orchestration solution for the Microsoft Azure cloud. The service is available in all Azure regions and can deploy with nearly all Azure VM sizes available, including those with specialized hardware such as GPUs and Infiniband/RDMA interfaces.

Azure Batch is a managed job scheduling solution where the underlying compute resources are allocated as pools. Pools may be comprised of Linux or Windows VMs and any number of dedicated and low priority nodes and can be automatically scaled through autoscale formulas. An Azure Batch Job can be thought of as a queue for tasks. Each Azure Batch Job can be assigned to a pool. Any number of tasks can be assigned to jobs, including multi-instance tasks (i.e., multi-node) which support MPI jobs. Azure Batch also supports containerized execution.

For a full introductory treatment of Azure Batch, see the Azure Batch Technical Overview. The service team also provides a GitHub repository for issue tracking, feature requests, and change logs.

416 questions
2
votes
1 answer

Azure Batch job release task not running

We execute a web request to our service from a job release task since azure batch (as far as I am aware) does not provide hooks for task completion to notify our service of a task completion. We do not want to poll for task completion. We found that…
Richard
  • 69
  • 6
2
votes
1 answer

RelativeMountPath under Azure Batch Pool

What value do I need to provide for RelativeMountPath to mount a file share to the batch pool with windows compute…
Mike143
  • 163
  • 3
  • 14
2
votes
2 answers

Accessing Azure Key Vault from Azure Batch

I am trying to access secrets in my Azure Key Vault from VMs running in my Azure Batch node pool. However, I keep running into the exception: Exception Message: Tried 1 certificate(s). Access token could not be acquired. Exception for cert #1 with…
apleroy
  • 587
  • 1
  • 6
  • 13
2
votes
1 answer

Getting 'This operation is forbidden on unbound objects' when trying to monitor an Azure Batch task

I want to execute a simple task in Azure Batch, to wait until it completes and to get the result: using (var client = _CreateBatchClient()) { var monitor = client.Utilities.CreateTaskStateMonitor(); var task = new…
psfinaki
  • 1,814
  • 15
  • 29
2
votes
1 answer

Azure Batch VNet Integration - Failing to provision nodes

I'm trying to provision a Pool of Nodes within Azure Batch and have the nodes join an existing VNet that has other applications sitting inside of it, to no avail... I have a subnet created and delegated to Azure Batch. The pool tries to resize and…
David C
  • 501
  • 1
  • 4
  • 16
2
votes
3 answers

PS Script to manage Azure Batch Application Version Retention

We have an Azure Devops release that builds our repo, packages it, and deploys it to our Azure Batch service with the following PowerShell script: New-AzureRmBatchApplicationPackage -AccountName "$(BatchAccountName)" -ResourceGroupName…
Jeremy Holovacs
  • 22,480
  • 33
  • 117
  • 254
2
votes
1 answer

Azure Batch - Automatically delete completed tasks

Is there a way to automatically delete task in job after successful execution? I'm setting retention time to 24 hours and it correctly deletes the data associated with a task, but not the task itself. I know this is the expected behavior. But how…
270F
  • 119
  • 2
  • 12
2
votes
1 answer

Azure Batch environment variables security

I'm not sure how secure are environment settings in Azure Batch. Do I need to use certificates? I am running tasks in Azure Batch, currently passing storage token with EnvironmentSetting. I cannot find any details in documentation on how secure is…
cpt-planet
  • 133
  • 5
2
votes
1 answer

Azure Batch: Do I get charged after deleting job but still keep pool?

I found it takes a few minutes to create pool and prepare the job, I am thinking to keep the pool to get rid of the overhead. Thanks Lidong
lidong
  • 556
  • 1
  • 4
  • 20
2
votes
1 answer

Does Azure Batch support .Net Core in Azure Data Factory v2?

I've been trying to get even a simple "Hello World" .Net Core 2.x Console App running via ADF v2 Custom Activity/Batch Service but with no luck. It does seem to work when I provision a WindowsServer and run a .Net Framework 4.5.x Console App though.…
Dakhath
  • 103
  • 1
  • 10
2
votes
4 answers

Azure Data Factory - Clean Up Batch Task Files

I'm working with Azure Data Factory v2, using a Batch Account Pool w/ dedicated nodes to do processing. I'm finding over time the Batch Activity fails due to no more space on the D:/ temp drive on the nodes. For each ADF job, it creates a working…
Mike R
  • 631
  • 5
  • 18
2
votes
1 answer

FileUploadMiscError while persisting output file from Azure Batch

I'm facing the following error while trying to persist log files to Azure Blob storage from Azure Batch execution - "FileUploadMiscError - A miscellaneous error was encountered while uploading one of the output files". This error doesn't give a lot…
Shashank Verma
  • 369
  • 1
  • 14
2
votes
2 answers

Automatically spawn an Azure Batch AI job periodically

I want to automatically start a job on an Azure Batch AI cluster once a week. The jobs are all identical except for the starting time. I thought of writing a PowerShell Azure Function that does this, but Azure Functions v2 doesn't support PowerShell…
haroba
  • 2,120
  • 4
  • 22
  • 37
2
votes
1 answer

Invoking Ruby Scripts or Executables on on-premise server using Azure Data Factory v2?

How can I invoke Ruby scripts or executable(s) on on-premise server using Azure Data Factory v2 (ADF)? In my view, it falls under the category of Custom Activity and ADF as of now supports custom activity only via Azure Batch and Azure HDInsight.…
rh979
  • 657
  • 1
  • 5
  • 13
2
votes
3 answers

How to pass application path to task in Azure batch

I am using Azure batch. I have an exe which I need to execute on compute nodes. I am using this path in the azure portal in my task window "cmd /c %AZ_BATCH_APP_PACKAGE_MyAppCreateRG%\CreateRG.exe -args HelloRG eastUs" But I am getting an error: The…
user1037747
  • 1,307
  • 2
  • 18
  • 38
1 2
3
…
27 28