0

Assume you have two queues, queue A with some new hardware, and queue B with old hardware. Further, both queues have the same number of nodes and slots for SGE jobs, e.g. 10 slots per queue.

Now I submit 10 jobs. If both queues are empty and the nodes have the same load, I guess 5 of the jobs will run in queue A and 5 of them in queue B. But since B has older hardware, the jobs will take longer to finish.

Is there a way to ask SGE to try and "fill" the fast nodes in queue A first before "backing off" to the old queue B? I'm asking about both user and admin side of the question.

1

I think you may misunderstand the queues, or you don't have sufficient information. SGE won't split a submission between different queues, you generally submit to one queue. So you'd have control in your job submission were you want to submit. You could simply script a submission of all jobs to queue A, a check on qstat to see what jobs are holding vs running, then resubmit those jobs to queue B and then cancel the jobs holding in queue A.

You could also just have it all in one queue, which is what I would do. Look at: https://www.icts.uiowa.edu/confluence/display/ICTSit/SGE+adaptive+sequence+scheduling for an overview of how you can set the preference / order for next job scheduling with some scripting - though in that example they are also doing overlapping queues with priorities, which you need not do (unless you wanted to)...

To set the fill sequence, see http://gridscheduler.sourceforge.net/htmlman/htmlman5/sched_conf.html to change from load based to sequence based fills. You would then make the newer nodes have earlier sequence numbers than the older nodes.

1
  • Thank you, the last comment on the seqno fill method was very helpful!
    – Pavel
    Mar 9 '15 at 10:13

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.