0

Assume you have two queues, queue A with some new hardware, and queue B with old hardware. Further, both queues have the same number of nodes and slots for SGE jobs, e.g. 10 slots per queue.

Now I submit 10 jobs. If both queues are empty and the nodes have the same load, I guess 5 of the jobs will run in queue A and 5 of them in queue B. But since B has older hardware, the jobs will take longer to finish.

Is there a way to ask SGE to try and "fill" the fast nodes in queue A first before "backing off" to the old queue B? I'm asking about both user and admin side of the question.

Pavel
  • 1,038
  • 1
  • 11
  • 30

1 Answers1

1

I think you may misunderstand the queues, or you don't have sufficient information. SGE won't split a submission between different queues, you generally submit to one queue. So you'd have control in your job submission were you want to submit. You could simply script a submission of all jobs to queue A, a check on qstat to see what jobs are holding vs running, then resubmit those jobs to queue B and then cancel the jobs holding in queue A.

You could also just have it all in one queue, which is what I would do. Look at: https://www.icts.uiowa.edu/confluence/display/ICTSit/SGE+adaptive+sequence+scheduling for an overview of how you can set the preference / order for next job scheduling with some scripting - though in that example they are also doing overlapping queues with priorities, which you need not do (unless you wanted to)...

To set the fill sequence, see http://gridscheduler.sourceforge.net/htmlman/htmlman5/sched_conf.html to change from load based to sequence based fills. You would then make the newer nodes have earlier sequence numbers than the older nodes.

jmp242
  • 688
  • 3
  • 15