Having issue with slurm. error: "no such file or directory"

Question

I'm trying to run a slurm script using sbatch <script.sh>. However, despite checking my path variable multiple times, i get a file not found error. Moreover I think this has to do with my go environment but I also get a "cannot import absolute path" error. I'm not sure what the issue is. I have attached my slurm configuration file as well as the error output below

#!/bin/bash
#SBATCH --partition production
#SBATCH --nodes=1
#SBATCH --ntasks-per-node=1
#SBATCH --cpus-per-task=1
#SBATCH --time=5:00:00
#SBATCH --mem=2GB
#SBATCH --job-name=myTest
#SBATCH --mail-type=END
#SBATCH --mail-user=atd341@nyu.edu
#SBATCH --output=slurm_%j.out

module purge
module load go/1.17
##RUNDIR=${SCRATCH}/run-${SLURM_JOB_ID/.*}
##mkdir -p ${RUNDIR}
DATADIR=${SCRATCH}/inmap_sandbox
cd $SLURM_WORK_DIR
source $DATADIR/setup.sh
go run $DATADIR/

Here is the output:

/var/spool/slurmd/job16296/slurm_script: line 19: /inmap_sandbox/setup.sh: No such file or directory import "/inmap_sandbox": cannot import absolute path

I have tried checking my path variable and making sure I'm following the correct path. For reference by directory structure is /scratch/inmap_sandbox. I'm trying to run the sbatch file in the /scratch directory

Dan Bonachea · Answer 1 · 2022-12-16T22:40:53.873

Offhand it appears the ${SCRATCH} variable might not be set inside the environment running the script. Try explicitly setting that to /scratch?

Once you get past that problem, note that if this batch script is running on a compute node that is separate from the frontend node you are using interactively, then they might not both mount the same ${SCRATCH} file system (or possibly mount it in different places).

Consult the system documentation to find out which file systems are shared between the frontend and the compute nodes. You might even need to pass SLURM capability options to request certain shared filesystems. In the absence of documentation, comparing the output of mount on the frontend and from within the batch script might be helpful. More specifically, add the mount command on a line by itself early in your batch script, and compare the output it generates to the output of the same command on the frontend.

Having issue with slurm. error: "no such file or directory"

1 Answers1