Stack Overflow
Stack Overflow

Questions tagged [azure-batch]

The azure-batch tag should be used to ask questions regarding all aspects of the Azure Batch service including job scheduling, compute node provisioning, and pool orchestration. Extended ecosystem questions are also welcome that are related to Azure Batch accelerator offerings, Azure Portal Batch blades, and Batch Explorer.

Azure Batch is a job scheduling and compute management orchestration solution for the Microsoft Azure cloud. The service is available in all Azure regions and can deploy with nearly all Azure VM sizes available, including those with specialized hardware such as GPUs and Infiniband/RDMA interfaces.

Azure Batch is a managed job scheduling solution where the underlying compute resources are allocated as pools. Pools may be comprised of Linux or Windows VMs and any number of dedicated and low priority nodes and can be automatically scaled through autoscale formulas. An Azure Batch Job can be thought of as a queue for tasks. Each Azure Batch Job can be assigned to a pool. Any number of tasks can be assigned to jobs, including multi-instance tasks (i.e., multi-node) which support MPI jobs. Azure Batch also supports containerized execution.

For a full introductory treatment of Azure Batch, see the Azure Batch Technical Overview. The service team also provides a GitHub repository for issue tracking, feature requests, and change logs.

416 questions
1
vote
1 answer

Set-AzBatchPool starttask resourcefiles A constructor was not found Microsoft.Azure.Commands.Batch.Models.PSResourceFile

I'm trying to set Azure Batch pool start task via powershell and Az module . Need to set ReosurceFiles but cannot create object for it System.Collections.Generic.IList[Microsoft.Azure.Commands.Batch.Models.PSResourceFile] New-AzBatchResourceFile -…
T SOLvic
  • 11
  • 2
1
vote
1 answer

Azure batch does not refresh cached containers in the pool

I have a container image prefetched on the Azure batch account pool, but I notice that the pool doesn't auto refresh the container when a new image is uploaded to the container registry, even though I have asked Batch to cache the latest container…
sargeMonkey
  • 606
  • 1
  • 7
  • 23
1
vote
1 answer

Difference between Azure Virtual Machine Scale Set and Azure Batch

I am new to Azure. Does anyone know the basic difference between VMSS and Azure Batch. I have studied the material on the Microsoft website, and there indeed is a difference, but the terms are complicated for me as a beginner. Can anyone please…
Murk
  • 75
  • 1
  • 8
1
vote
0 answers

Bad Request Creating Azure Batch user subscription pool using c# with vnet

I was trying to create a multi node pool in Azure Batch (User Subscription) using C#. I have assigned the RBAC role of "Virtual Machine Contributor" to the Vnet for "Microsoft Azure Batch" along with it's already existing Contributor role. While I…
Mahesh
  • 13
  • 2
1
vote
1 answer

BatchErrorException: The specified operation is not valid for the current state of the resource

Scenario: I have a job created in azure batch against a pool. Now, I created another pool and want to point my job to the newly created pool. I use the azure-batch SDK to write the following piece of code import azure.batch.batch_service_client as…
Sandeep Menon
  • 45
  • 5
1
vote
2 answers

How to access Azure Database from Azure Batch?

I have a Java process inside a Docker container inside an Azure Batch Task. The Java process is trying to simply establish a JDBC database connection to the Azure Database for MySQL service. The connection error includes 'Client with IP address…
Oozman
  • 41
  • 4
1
vote
1 answer

Failure to unzip application packages in Azure Batch

I followed the guidance in this wiki page to create an application package for my Azure Batch pool, but now my nodes are stuck in an unusable state because it fails to unzip. I can't find anything in the documentation that talks about what kind of…
GeneSavvy Chris
  • 35
  • 4
1
vote
1 answer

Azure batch windows container workloads remove quotes from provided command line

When using windows container workloads on Azure Batch quotes are stripped from command line arguments if they do not contain a space. We are using the newest C# SDK 13.0.0 and node VMs SKU windows server 2019 with containers. Repro: Create job and a…
Richard
  • 69
  • 6
1
vote
1 answer

Set azure batch cluster to run in specific hours

Is it possible to run Azure Batch cluster only in specific hours? What variables should I use for it? Eg. from 6 am to 6pm. Will cluster be available to run in other hours if any tasks will be forced on it?
Jack n J
  • 206
  • 3
  • 14
1
vote
1 answer

Azure Batch Application Insights

I'm trying to get application insights to work in azure batch job/task while running through a .net core 3.1 console application. When I run the console app through Visual Studio debug or directly on a node in the batch pool, I get all the…
Azbami
  • 11
  • 2
1
vote
1 answer

Azure Batch and Determine the Path of the Downloaded EXE on the Compute Node in c#

I am using Azure Batch. I am writing a c# application that uploads files to an Azure storage account. The code then creates a pool and tasks - this is all working fine except for one thing. The task has a commandline which is executed on each node…
Ian Carrick
  • 242
  • 3
  • 18
1
vote
1 answer

Azure Batch Job Schedule - notification after recurrence job completed/failed

I am trying to use Azure Batch Job Schedule in my application with .Net core. I want to get some notification/event trigger once the recurrence job is completed/failed in the job schedule so that I can copy output files to storage and send email to…
Vikas Pawar
  • 135
  • 1
  • 2
  • 11
1
vote
0 answers

How to use Azure Data Lake Gen2 for Azure Batch Service?

For Processing a huge amount of Data, we wanted to use Azure Data Lake Gen2 Storage with Azure Batch. Here's what I have tried: I created Pool, Job, and Uploaded File to the Data Lake file system (reference taken from Microsoft Docs). When the Batch…
Gour Gopal
  • 581
  • 7
  • 22
1
vote
2 answers

Azure Batch mounting blob storage on pool results in error "Counld not install package"

I have an Azure Batch pool where I have mounted three blob storage containers. It works, however, when the nodes are booting for the first time they get the following error: Looking in the logs it seems the nodes have trouble installing the…
niknoe
  • 323
  • 5
  • 14
1
vote
1 answer

Docker run python script cant find locally installed module

For context, this problem relates to a docker image that will be run using azure batch. Here is the Dockerfile, in full: FROM continuumio/miniconda3 ADD . /pipegen ADD environment.yml /tmp/environment.yml RUN conda env create -f…
JmeCS
  • 497
  • 4
  • 17
1 2 3
…
27 28