Is it possible to configure the directory for sbatch's default output file?

Question

Is there some way to configure an alternative default directory (other than the current directory) for sbatch to put the file slurm-%j.out (or slurm-%A_%a.out) that it generates when the -o is not specified?

My goals here are to have a convenient/low-maintenance way to

consolidate all default output files generated by sbatch in one place; and
avoid cluttering the current directory with such files.

I had hoped to find something like a SLURM_DEFAULT_OUTPUT_DIRECTORY environment variable to take care of this, but if there is such a variable, I managed to miss it.

Is there some other mechanism to achieve the desired results?

(FWIW, the only precedent that comes to mind for the functionality I'm after is Emacs's backup-directory-alist, which tells Emacs were to put the backup files it generates by default. This feature is more sophisticated than what I'm after, because it supports multiple such directories, to be used according to patterns in the original filenames.)

score 15 · Answer 1 · edited Oct 21 '19 at 14:58

15

Pass a log file into a folder as follows in bash script:

#SBATCH -o ./Report/output.%a.out # STDOUT

This will put the output files in "Report" folder next to your batch script.

edited Oct 21 '19 at 14:58

Farid Alijani

839
1
7
25

answered Apr 20 '18 at 02:38

milihoosh

547
5
10

1

does this command create `Report` folder is does not already exist? – Farid Alijani Oct 21 '19 at 11:10
1

From my tests it would appear not. – davidrpugh Oct 29 '19 at 06:40
No, it does not! – milihoosh Nov 01 '19 at 21:04
You need to create the "Report" folder first!!!! The ouput file will not show up if you didn't create one. – jonijeng Jul 27 '22 at 07:33

Matheus Torquato · Answer 2 · 2022-04-27T08:53:17.220

The .sh file below will save the slurm output .out file at

"/home/YOUR_USER/lustre/PROJECT/output/%j.out"

where %j is the jobid of the running job as explained here.

So, in the output directory mentioned above you'll find something similar to this:

399937.out

#!/bin/bash
#SBATCH --job-name=JOB_ID_NAME
#SBATCH --time=4-00:00          # adjust this to match the walltime of your job
#SBATCH --cpus-per-task=28      # adjust this if you are using parallel commands
#SBATCH --mem=1000              # adjust this according to the memory requirement per node you need
#SBATCH --mail-user=YOUR-EMAIL  # adjust this to match your email address
#SBATCH --mail-type=ALL
#SBATCH --output=/home/YOUR_USER/lustre/PROJECT/output/%j.out

I tried this with `-o` and it tries to find "/home/YOUR_USER/lustre/PROJECT/output/%j.out" in the current directory that the bash script is rather than go to the specified directory. — Gooby, Sep 02 '22 at 21:18

score 1 · Answer 3 · answered Mar 20 '18 at 14:03

1

An other option is to use a Job Submit Plugin.

You can write such a plugin either in C or in LUA so it forces the output file (default base name but your own directory in your case) if not already set by the script nor the command line.

answered Mar 20 '18 at 14:03

Gilles Gouaillardet

8,193
11
24
30

score 0 · Answer 4 · answered Mar 15 '18 at 08:42

I faced the same problem and my solution was to copy the submission script into the directory designated for the output. I then ran the script located in the output directory. The directory of the code intended to be submitted with SLURM can be communicated dynamically to the job script via an environment variable. The code directory can also be set as the working directory before executing the code. I hope somebody finds this solution helpful.

Is it possible to configure the directory for sbatch's default output file?

4 Answers4