SGECluster in multiple queues

Asked Dec 18 '19 at 11:24

Active Dec 18 '19 at 11:24

Viewed 74 times

I'm using dask.distributed to launch jobs on a SGE cluster (https://jobqueue.dask.org/en/latest/generated/dask_jobqueue.SGECluster.html#dask_jobqueue.SGECluster) via dask.bags and/or dask.delayed.

Everything works nicelly. However, i may have some dask.delayed that should run in a specific queue on my SGE cluster (due to GPU availability for instance). How can I work this out on dask?

In other words, how can I define a dask_jobqueue.SGECluster with multiple queues and/or different resource specs? dask_jobqueue.SGECluster allows me to configure one cluster setup only (one queue, one resource spec, etc..).

Thanks

asked Dec 18 '19 at 11:24

Tiago Freitas Pereira

I believe that there is an open issue about this on github at https://github.com/dask/dask-jobqueue/issues somewhere – MRocklin Dec 21 '19 at 04:02
Well, I couldn't find similar request. – Tiago Freitas Pereira Jan 03 '20 at 09:13
Anyone willing to help? Thanks – Tiago Freitas Pereira Jan 08 '20 at 13:12
I think this issue seems relevant: https://github.com/dask/dask-jobqueue/issues/304 – quasiben Jan 08 '20 at 14:48

SGECluster in multiple queues

0 Answers0