How to loop through a text file using sbatch jobs

Question

I have a text file containing some lines of text and I want to pass each line of the file as an argument in my python script. The issue I'm having is reading the file. If I can read the file, I should be able to pass each line of the file as argument for my python job submission. here is what I've tried

#!/bin/bash
#SBATCH -N 1
#SBATCH -p RM-shared
#SBATCH -t 1:00:00
#SBATCH --ntasks-per-node=64


set file = \myfolder\sample.txt

for /f "tokens=*"  %%val in (%file%) do(
  echo %%val
  python3 test.py %%val
  )

When I run the batch submission of this job, and I check the slurm.out file, I see an error like this "unexpected error near '"tokens="'

Please does anyone know how I can read from a text file and pass it as an argument in my python script?

score 0 · Answer 1 · answered Jun 03 '22 at 12:15

Slurm expect the syntax of the submission script to be a valid (Linux) shell script, typically Bash (hence the shebang you set as the first line). But your script does not use Bash syntax but rather Windows Batch script syntax.

The Bash interpreted is confused by the %% syntax ; hence the error message "unexpected error near '"tokens="'.

You will have to translate the script into valid Bash.

How to loop through a text file using sbatch jobs

1 Answers1