[GE users] Using gridengine to administer the cluster
Michael.James at csiro.au
Wed Aug 30 01:16:58 BST 2006
[ The following text is in the "iso-8859-1" character set. ]
[ Your display is set for the "ISO-8859-10" character set. ]
[ Some special characters may be displayed incorrectly. ]
On Tue, 29 Aug 2006 11:50 pm, Daniel Templeton wrote:
> An easier way to submit once to each host would be
> to set up a consumable with a value of 1 for each host.
> All you'd have to do then is submit n jobs,
> each requesting the consumable, where "n" is the number of hosts.
On Tue, 29 Aug 2006 11:28 pm, Chris Dagdigian wrote:
> I second Joe's recommendation regarding pdsh.
> There are several tools designed to automate
> the execution of commands across clusters
> (including groups and subgroups of nodes)
> and they all work very well for admin centric tasks.
> Grid Engine may not be the best tool for this particular job.
Thanks to everybody for the excellent responses,
I must look into pdsh and pdcp.
But if I start using external commands,
how do I schedule them into gridengine?
ie: On a fairly full local data partition I want to run this:
biofile> rsync -a --delete /nfs/master/data/ /data/
"biofile" is the owner of the biological databases.
At present I have to wait for a quiet moment,
(or make one) and then run the job.
Otherwise I risk SGE sending a blast on nr while it's being updated.
The attraction of setting a consumable and slots
is that I can schedule the update and it waits
until it can monopolise each node and do its job undisturbed.
Clever scheduling would even mean I could set a limit
that only a third of my cluster are allowed
to take themselves offline for maintenance at any one time.
Michael James michael.james at csiro.au
System Administrator voice: 02 6246 5040
CSIRO Bioinformatics Facility fax: 02 6246 5166
No matter how much you pay for software,
you always get less than you hoped.
Unless you pay nothing, then you get more.
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users