Questions tagged [dask-jobqueue]

I have access to a cluster running PBS Pro and would like to keep a PBSCluster instance running on the headnode. My current (obviously broken) script is: import dask_jobqueue from paths import get_temp_dir def main(): temp_dir =…

I am running into some errors when trying to set up my own client using jobqueue PBS Cluster instead of using a default local cluster (i.e., client = Client()). When setting the default, my own modules were recognized, but I realized my workers in…

I'm trying to run dask on an HPC system that uses NFS for storage. As such, I want to configure dask to use local storage for scratch space. Each cluster node has a /scratch/ folder that all users can write into, with instructions to put scratch…

I have a PBS job script with an executable that writes results to out file. ### some lines PBS_O_EXEDIR="path/to/software" EXECUTABLE="executablefile" OUTFILE="out" ### Copy application directory on compute node [ -d $PBS_O_EXEDIR ] || mkdir -p…