0

I am new to SBATCH and bash. I have a simple script to submit the job and I am trying to get the parent directory of the current working directory. It seems from the documentation that '$(dirname $PWD)' is the way to do it. But it always returns null or is empty. I have tried very different versions of it such as '$(dirname '$PWD')' or just $(dirname $PWD). They all result in null. I am suspecting missing something very basic. Any recommendations on what I should check to resolve or debug this better will be helpful. Here is the sample code (simplified)

#!/bin/bash

sbatch <<EOT
#!/bin/bash
#SBATCH --nodes=1
#SBATCH --cpus-per-task=$2
#SBATCH --partition=express
#SBATCH --time=01:00:00
#SBATCH --mem-per-cpu=4G
#SBATCH --output=$1_%A_%a.out
#SBATCH --array=0-4%5
#SBATCH --error=%A_$1_%a.err
#SBATCH --constraint=broadwell

PARENT_DIR="$(dirname $PWD)"
echo "Main directory for submit-test-express script is: $PARENT_DIR"

echo "Current drectory is: $PWD"

echo "Job array ID: $SLURM_ARRAY_JOB_ID , sub-job $SLURM_ARRAY_TASK_ID is running!"


python /path_to_code/code.py $1 $2 $3 $SLURM_ARRAY_TASK_ID /output_folder/ > output_file-$SLURM_ARRAY_JOB_ID-$SLURM_ARRAY_TASK_ID.out

EOT

The code is wrapped in EOT so that the arguments can be passed from other scripts. Also note that the second echo statement prints the current working directory correctly. But the first echo statement returns blank.

RforResearch
  • 401
  • 8
  • 16
  • Can you log into the machine that will execute this and issue a simple `$ dirname $PWD` on its shell ? Does this print the correct value ? – Niloct May 02 '21 at 23:04
  • Thanks for the comment. I am running it on a discovery cluster. Just tried {dirname $PWD} and it gave me the correct path. – RforResearch May 02 '21 at 23:06
  • In case it is helpful, I also issued a simple 'echo $(dirname $PWD)' and it gave the correct output too. – RforResearch May 02 '21 at 23:12

1 Answers1

2

Since you are using $PARENT_DIR unscaped your variable gets evaluated before passing the full heredoc to sbatch, try to escape $ like: \$PARENT_DIR

jonathadv
  • 346
  • 2
  • 10
  • Yes! You are right. In the echo statement, I just added \$PARENT_DIR and it prints correctly. That might explain why even my SLURM_ARRAY_TASK_ID is also empty – RforResearch May 02 '21 at 23:18
  • 3
    @RforResearch To escape the entire here-doc, you can quote the delimiter: `<< 'EOT'`. – Benjamin W. May 02 '21 at 23:27
  • Does that mean it will also work for $SLURM_ARRAY_TASK_ID as well when passed as an argument? and also do you mean I put the 'EOT' in the beginning and the end? – RforResearch May 02 '21 at 23:35