AWS Batch enables you to run batch computing workloads on the AWS Cloud. Batch computing is a common way for developers, scientists, and engineers to access large amounts of compute resources. AWS Batch removes the undifferentiated heavy lifting of configuring and managing the required infrastructure.
Questions tagged [aws-batch]
420 questions
0
votes
2 answers
Cannot assign UUID to bash variable
I want to script termination of jobs on AWS Batch and I am running into problems while assigning UUIDs to bash variables.
I have a text file named 'LastAndFinal_job_id_list.txt with two columns as follows:
cbac858b-3884-41c7-8bb7-97eeca8339ae …
Sri Hari Vignesh
- 256
- 4
- 11
-1
votes
1 answer
Choosing between AWS Batch(Fargate Compute Environment) with EventBridge Scheduler and AWS Fargate with Scheduler for a Java Task Scheduler
I have a Java task scheduler that runs approximately 6 hours per day on AWS. I'm considering two deployment options and seeking advice on which one is more suitable for my use case:
Option 1: AWS Batch(Fargate Compute Environment) with EventBridge…
Sivaram Rasathurai
- 5,533
- 3
- 22
- 45
-1
votes
1 answer
AWS batch to execute on prem batch code through a rest call or via sending event
Can I trigger on premises batch from AWS without migrating batch code.
is it possible through rest call or queue?
I don't want to move batch code to AWS instead just call batch code from AWS
Hari
- 1
-1
votes
1 answer
Running multi-node parallel job in AWS Batch using R
I am trying to build a multi-node parallel job in AWS Batch running an R script. My R script runs independently multiple statistical models for multiple users. Hence, I want to split and distribute this job running on parallel on a cluster of…
Telis
- 311
- 1
- 9
-1
votes
2 answers
I am not able to pass parameter in aws batch job?
I have a Dockerfile with the following lines:
FROM python
COPY sysargs.py /
CMD python sysargs.py --date Ref::date
And my python file sysargs.py looks like this:
import sys
print('The command line arguments are:')
a = sys.argv[1]
print(a)
I just…
reddaveni rahul
- 1
- 2
-1
votes
1 answer
How many events will be fired if we insert data into Dynamo DB table in the form of batches?
Batch job inserts data into Dynamo DB table 'T' in the form of batches of size 500 and there is one trigger on table 'T', Will DynamoDB fire event for each of the 500 records separately or single event for the entire batch?
Asraar Wani
- 70
- 7
-1
votes
1 answer
Multifunctional SpringBoot JAVA Application (REST/BATCH/LAMBDA)
I have a java spring boot application that runs a job to upload data to Database after polling a message from SQS and this application also contains a REST API over that same database.
Now I need to decouple the upload functionality and REST…
Tarun Kundhiya
- 156
- 1
- 9
-1
votes
1 answer
Trigger event when specific 3 AWS jobs are completed
I have submitted 3 jobs in parallel in AWS Batch and I wanted to create a trigger when all these 3 jobs are completed.
Something like I should be able to specify 3 job ids and can update DB once all 3 jobs are done.
I can do this task easily by…
Saurabh Sharma
- 463
- 2
- 7
- 20
-1
votes
2 answers
Amazon ECS agent on ubuntu not starting
I am currently trying to build a custom ubuntu ami for AWS batch and following the document mentioned here
https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-agent-install.html
However when I try to start the docker agent on that…
Shrikar
- 840
- 1
- 8
- 30
-1
votes
1 answer
How to use AWS batch in java class(lambda function) to submit batch job
I read the output from stepfunctions using envContainerOverrides and then I am calling my batch job with this nodejscode: Here I am reading the environment variables which is passed to the batchjob.
Stepfunctions…
nad87563
- 3,672
- 7
- 32
- 54
-2
votes
2 answers
What is the easiest way to launch parallel jobs on AWS?
My use case is as follows:
I have a python script which:
1. reads a file from S3
2. processes the file and outputs a new file
3. saves the output file to S3 (or maybe a database)
The python script has some dependencies which are managed via…
Henry Henrinson
- 5,203
- 7
- 44
- 76
-3
votes
1 answer
Local network batch
I have been playing around with azure batch for a project at work.
My work has 100-1000s of computers running win10 mostly chewing over Outlook and not much else.
Is there some framework that can leverage 'idle' CPU time in this environment?
Seems a…
Jon
- 2,280
- 2
- 25
- 33
-4
votes
1 answer
Can we send multiple messages to a single SNS topic using a single AsyncPublish sns call?
I have a use case where i need to send one or more messages to a single sns topic in one asyncPublish sns call. Is there a possible way by which i can do that ?
