[GE users] SGE control of heterogeneous clusters

hjmangalam harry.mangalam at uci.edu
Thu Jun 4 18:39:56 BST 2009

I'm not an SGE nube anymore, but I wouldn't call myself an expert by 
any means. We have a semi-heterogeneous cluster running on 6.2 in a 
univ environment:

* started with mixed 32/64b nodes; now almost all 64b nodes.
* ~1/2 running in a private net (CentOS 5)
* ~1/2 running on a public net (CentOS 5)
* a few large specialty nodes in the private net (Ubuntu 8.10)
* all sharing SGE_ROOT, /home, /shared_apps, versioning 
    and arch handled by 'module'
* login by LDAP-supplied ID.
* all more or less happily being fed by the same instance of SGE 6.2.

The response from users is fairly good, to the extent that a few of 
them have inquired as to the possibility of adding their own clusters 
to this setup to contribute their unused cycles and harvest others' 
when they need it.

I can see a fair number of problems related to trying this.

- they would have to mount SGE_ROOT (not a problem)

- they would have to share /home (possibly a larger problem)

- the different clusters would have different upgrade timing for apps, 
shared libs, etc.  This would be a problem if they were going to try 
to share apps across clusters, but not if they were using apps from 
their own clusters or supply their own app/lib trees via their $HOME.

- some security issues related to more users, different attention to 
security details, different upgrade schedules, etc.

- initial problem of rationalization of UIDs/GIDs to the LDAP model 
(or keeping them separated in some way?).

However, from what I'm seeing on my current tripartite (tho simple) 
cluster, it seems possible.  

Is there a consensus on how well this works when trying to scale out 
to include other clusters?

thanks Harry

Harry Mangalam - Research Computing, NACS, Rm 225 MSTB, UC Irvine
[ZOT 2225] / 92697  949 824-0084(o), 949 285-4487(c)
MSTB=Bldg 415 (G-5 on <http://today.uci.edu/pdf/UCI_09_map_campus.pdf>
Don't worry about people stealing your ideas. If your ideas are any 
good, you'll have to ram them down people's throats.  <Howard Aiken>


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list