[GE users] question about queues and limitations to specific nodes

reuti reuti at staff.uni-marburg.de
Wed May 12 22:20:48 BST 2010


Hi,

Am 12.05.2010 um 22:28 schrieb jnorris:

> Here is the situation:
>
> I have 66 nodes total in my cluster - I need to allow select users to
> access the first 8 nodes and all users access the remaining nodes.
>
> The first 8 nodes I put into a queue ib.q and the rest in  
> ethernet.q  I
> set the exclude user list on IB to all of the user lists of those that
> should not be allowed and the user list to the users that are allowed.

is the list of the "extra-users" a subset of the "users" one? When a  
user is in both, access will be denied.


> With ethernet.q  I have all user list in users and NONE for exclude  
> users.
>
> Then I did some testing with a simple hello script that keeps track of
> what node its running on and prints the message:
>
> Hello from process 0 on c1
> Hello from process 1 on c2
> Hello from process 2 on c3
> Hello from process 3 on c4
> Hello from process 4 on c5
> Hello from process 5 on c6
> Hello from process 6 on c7
> Hello from process 7 on c8
>
>
> I set the #$ -q ethernet.q  in the script and set the run as:
>
> mpiexec.hydra  -f /opt/mpdd_tmp/hydra.hosts  -n 10 ./hello

Is Hydra now final? When I checked it last, it was only a beta and I  
didn't look further into it. AFAIK Hydra is not tightly integrated  
into SGE:

https://trac.mcs.anl.gov/projects/mpich2/ticket/443

So we would need to use rsh/ssh to be directed to a `qrsh -inherit`.  
For now it looks like it's using any node and not the granted ones.  
Which type of PE did you set up?


> The out put above is what I obtain.  Ok here are my questions:
>
> 1)  is this the best way to do this? - I am real newbie to sge

In some way this is also a personal taste how you would like to -  
let's say - visualize the usage. In principle you can also have:

- only one queue
- one hostgroup @ib_nodes for the first 8 nodes (to avoid specifying  
each machine [see two lines below])
- one users list called "others" which contains only the ordinary  
users w/o the special ones
- set in this queue: xuser_lists none,[@ib_nodes=others]

One more pitfall: when you attach the same PE to two queues you can  
also get a mixture of slots from both queues in your setup. A second  
PE hydra.ib, which is attached to

pe_list hydra,[@ib_nodes=hydra hydra.ib]

(or "pe_list hydra,[@ib_nodes=hydra.ib]" if ethernet jobs shouldn't  
run on the IB nodes at all)

some nodes, can help. And submit either "-pe hydra 4" or "-pe hydra.ib  
4" to choose the machines. Advantage: use "-pe hydra* 4" if you simply  
don't care what you get.

-- Reuti

> 2)  by the scenario above and considering that c1-c8 are the nodes  
> that
> should not be in this output - what am I doing wrong?
>
> Thank beforehand for any and all help.
>
>
>
> -- 
> Joseph Norris
> Application Developer & Server Administrator
> 209-228-4576
> jnorris at ucmerced.edu
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=257108
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net 
> ].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=257114

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list