4

I have a SLURM cluster and a RUNNING job where I have requested 60 threads by

#SBATCH --cpus-per-task=60

(I am sharing threads on a node using cgroups)

I now want to reduce the amount of threads to 30.

$ scontrol update jobid=274332 NumCPUs=30
Job is no longer pending execution for job 274332

The job has still 60 threads allocated.

$ scontrol show job 274332
JobState=RUNNING Reason=None Dependency=(null)
NumNodes=1 NumCPUs=60 NumTasks=1 CPUs/Task=60 ReqB:S:C:T=0:0:*:*

How would be the correct way to accomplish this?

Thanks!

Mike Nathas
  • 1,247
  • 2
  • 11
  • 29

1 Answers1

5

In the current version of Slurm, scontrol only allows to reduce the number of nodes allocated to a running job, but not the number of CPUs (or the memory).

The FAQ says:

Use the scontrol command to change a job's size either by specifying a new node count (NumNodes=) for the job or identify the specific nodes (NodeList=) that you want the job to retain.

(Emphasis mine)

damienfrancois
  • 52,978
  • 9
  • 96
  • 110