0

Before submitting one job, could I query one node's status, if the node is free, then submit it into the node; if not ,then submit the job into another free node. I want the job to run on the total node, not sharing with other jobs. Is there any way?

mining
  • 3,557
  • 5
  • 39
  • 66

1 Answers1

0

Since you've tagged this sungridengine I'll assume that's what you are using. Explicitly requesting by name a node that is currently empty doesn't guarantee that nobody will submit a job later that will be assigned to the same host.

To achieve this the admin needs to create an exclusive resource and associate it with each execution host. You can then request said resource to get the whole node.

William Hay
  • 2,148
  • 16
  • 20
  • Thanks, Sir. I write a bash script to submit my jobs one after one, sleep one second between the nearest two jobs. But, the jobs are often submit into one node, so this is very slow. Because these jobs have two stages, in the first stage, there are some data preprocessing, which is not parallel, and in the second stage, there are some parallel code to use multicore processing. So I think when the job sleep in one second, the SGE manager query the node is not busy, so it submit the other job into the node. So I want to check the node state before the SGE manager submit a job into it. – mining Oct 26 '13 at 08:36
  • If your jobs are parallel you could request a suitable parallel environment that uses up all the slots on a node. Again this is something the cluster administrator would have to set up. – William Hay Oct 26 '13 at 16:02
  • Thanks. But I am a common user, I think the cluster administrator didn't agree with me. – mining Oct 27 '13 at 00:04
  • Most grid engine cluster administrators have set up at least one PE for parallel jobs. You might be better off asking the administrator. Per your original question qhost -j -h will show any jobs on a node but it won't do you much good unless you are the only user on the cluster. If you are the only user then I would imagine the cluster administrator would be fairly co-operative. – William Hay Oct 28 '13 at 10:48
  • The cluster system just has fifty computing nodes, and each time I submit my jobs, I found the nodes are busy, and someone always takes more than 10 nodes, sometimes 20+ nodes, so my jobs are always waiting, So it is very boring. – mining Oct 28 '13 at 12:13