Stack Overflow
Stack Overflow

Questions tagged [sbatch]

sbatch submits a batch script to SLURM (Simple Linux Utility for Resource Management). The batch script may be given to sbatch through a file name on the command line, or if no file name is specified, sbatch will read in a script from standard input. The batch script may contain options preceded with "#SBATCH" before any executable commands in the script.

sbatch submits a batch script to SLURM (Simple Linux Utility for Resource Management). The batch script may be given to sbatch through a file name on the command line, or if no file name is specified, sbatch will read in a script from standard input. The batch script may contain options preceded with "#SBATCH" before any executable commands in the script.

231 questions
0
votes
0 answers

Slurm Exit Code 9: too much time between signals elapsed (32 seconds) - job killed

I am bootstrapping a panel of 2.7M using reghdfe and ppmlhdfe. I am using the Picotte cluster at Drexel, as this is computationally infeasible otherwise. When I run this, my job is killed because 1 of the iterations to compute an estimator takes…
Torin McFarland
  • 43
  • 6
0
votes
0 answers

HPC slurm srun setting ports listening like docker "-p" flag

I have an issue serving contents from a container (sqsh format) on worker node (remote cluster on perimeter) threw login node to my local workstation. there are software tools requires ssh tunnel to allow access to remote services like 1. visual…
Ori Kovacsi-Katz
  • 11
  • 3
0
votes
0 answers

How to use sbatch with a docker container that holds a snakemake pipeline?

I am building a Snakemake pipeline that I am going to put in a container. I want users to be able to pull the container using Apptainer or Docker. I would also like users to be able to run the container locally or using SBATCH. I know that there are…
Sam H
  • 45
  • 4
0
votes
0 answers

How to monopolize a GPU after submitting a job in slurm on a cluster

I have used the instructions such as 'nvidia-smi -c 3' or 'nvidia-smi -i 0 -c 3'. But they seem have no effect. The following is my script in slurm. But the GPU my job runs on is still shared with other people. #!/bin/bash #SBATCH -o…
BONNED
  • 23
  • 1
  • 5
0
votes
0 answers

How to read large size tab separated multiple files in R exe using SLURM?

I'm attempting to write a function that reads several tab-separated files (thousands) in SLURM via sbatch using R exe code, does some tasks, and then outputs tab-separated files. The program works effectively with files smaller than 2 GB but fails…
Debajyoti Kabiraj
  • 93
  • 7
0
votes
1 answer

Slurm Job Output File

I am running multiple jobs trying to have better performance results. To this end, I am generating a UUID and need the file name of the output to include this UUID. Currently, I am defining the name of the output as follows: #SBATCH…
ferdaous
  • 3
  • 2
0
votes
1 answer

Slurm SBATCH does not save all system output all a job failed

I am running a job that requires a large memory on a cluster using Slurm. I used the flags --output to save the system output. This would successfully save the system output if the job finishes without error. However, if the job encounters an…
ferris
  • 101
0
votes
1 answer

Rsync cannot connect to host through slurm

I am currently utilizing Slurm to schedule a data transfer using Rsync. I usually connect to the destination server using SSH with the command "ssh user_name@server" and it works without any issues. However, I am having an error when trying to use…
user1439599
  • 3
  • 2
0
votes
0 answers

Wait for a set of subprocess.Popen jobs to end (paralell execution)

I am trying to parallel a process executing subprocess.Popen un order to connect with sbatch, following the next squeme: >for i in jobs: > cmd=f'\ > sbatch --jobname={i} --mem=3G \ > --cpus-per-task=4 --output=logs/output/{i}.out \ > …
Andrea aguado
  • 1
  • 1
0
votes
1 answer

Is it possible to retrieve the output file defined via #SBATCH --output=slurm.out after deletion?

I accidentally deleted the output file of an sbatch job that is still running while deleting that of completed jobs. I checked for .local/share/Trash and this folder does not even exist. Any chance I could retrieve this file using sbatch…
Toonia
  • 61
  • 1
  • 7
0
votes
1 answer

How to SSH to a SLURM scheduler and execute SLURM commands using a Python script rather than the CMD to run a Python script?

Currently I am connecting to the SLURM scheduler and running the requested SLURM commands (such as sbatch and squeue) in the following manner (using Ubuntu 22.04): SSH to the SLURM job scheduler using the necessary credentials in the CMD Submit the…
AAA7
  • 3
  • 1
0
votes
0 answers

How to submit a Slurm job from local machine to GPU cluster?

I wanted to know is it possible to submit a Slurm job from my local machine without logging into the GPU cluster? For example: Everytime when I need to submit a job I have to copy the executable files to the GPU cluster directory and then sbatch the…
Mahesh
  • 556
  • 1
  • 3
  • 16
0
votes
0 answers

How to add ntasks value to slurm/sbatch output and error filenames

For a parallel program managed by slurm, I'm using the pattern described here https://slurm.schedmd.com/sbatch.html#lbAH to form the output files with the job name and job ids. However, I'm also running analysis on the program's runtime with the…
rigel
  • 485
  • 1
  • 6
  • 12
0
votes
1 answer

Running Slurm array jobs one per virtual core instead of one per physical core

we have a machine with 2x64-core CPU, each core consists of 2 virtual cores, so in htop we see 256 distinct (virtual) CPUs. We configured Slurm quality of service to better manage CPU usage per user. I.e. we have defined a --qos=cpus50 which, as far…
Paloha
  • 558
  • 6
  • 14
0
votes
0 answers

How to show the status of all job steps defined in a sbatch script, including those not yet created due to resource contention

I'm using SLURM sbatch to launch a bunch of parallel tasks in a cluster. The total amount of cores that I need to run all tasks in parallel exceeds the total amount of cores that my sbatch script asks for, so some job steps won't run until others…
fjs
  • 330
  • 2
  • 9
1 2 3
…
15 16