0

I am attempting to submit multiple jobs using a job array where I pass $SLURM_ARRAY_TASK_ID to Python's argparse, but $SLURM_ARRAY_TASK_ID does not print or produce the job files expected.

Here is the batch script that I am submitting:

#!/bin/bash 
#SBATCH --time=01:59:00
#SBACTH --array=1-2
#SBATCH --job-name=job_array
#SBATCH --output=log/job-%A-%a.out
#SBATCH --error=log/job-%A-%a.err
#SBACTH --array=1


echo 'Running code'

echo "Starting task" $SLURM_ARRAY_TASK_ID

python -u parse_test.py --var $SLURM_ARRAY_TASK_ID

echo 'Code done.'

The output of this job is:

Running code
Starting task
Code done.

With only one out and err file.

And I get an argparse error for no value passed for "var."

What I expect to get is:

Running code
Starting task 1
Starting task 2
Code done.

Along with two out and err files formatted job-###-$SLURM_ARRAY_TASK_ID.out(err) with the output(errors) from my python code.

I've searched the documentation for job arrays and I don't see what could be wrong here. Any help is much appreciated.

Alexander
  • 16,091
  • 5
  • 13
  • 29
asorlik
  • 11
  • 1
  • I'd change `parse_test.py` to just echo `sys.argv` to see if anything is done with that $$... – hpaulj Feb 22 '23 at 20:45
  • @hpaulj is looks like $SLURM_ARRAY_TASK_ID isn't even recognized since it isnt echoed in the first place. – asorlik Feb 22 '23 at 21:25

2 Answers2

0

You didn't call the array variable correctly. Doing echo $array in Bash will only print the first element in it.

Example:

$ ARRAY_TASK_ID=("task 1" "task 2")

$ echo $ARRAY_TASK_ID
task 1

$ echo ${ARRAY_TASK_ID[@]}
task 1 task 2

$ for ((i=0;i<${#ARRAY_TASK_ID[@]};i++)); do echo ${ARRAY_TASK_ID[$i]}; done
task 1
task 2

Try updating your python command to the following:

python -u parse_test.py --var "${SLURM_ARRAY_TASK_ID[@]}"

Though it's still unknown to me why your echo "Starting task" $SLURM_ARRAY_TASK_ID printed no ID. It indicates at least the first element in array is empty. Try inspecting the whole array using the examples above.

Taylor G.
  • 661
  • 3
  • 10
0

Change #SBACTH --array=1 to #SBATCH --array=1

Then echo should give you some value in the result.

j23
  • 3,139
  • 1
  • 6
  • 13