[GE users] resource management

Ara.T.Howard Ara.T.Howard at noaa.gov
Wed Apr 21 22:31:50 BST 2004


i have the following situation:

i am doing some processing where the output needs to be distributed evenly to
two large local disks, call them disk_a and disk_b.  each node in the cluster
has disks by these names of the same capacity.  in order for my job to run, it
must have a certain amount of free space on ONE of the two disks, furthermore
when running it needs knowledge of which disk it was allocated (it could
determine an appropriate disk to run on it's own, but not with out race
conditions i think...).

i understand generally how resource management works and have read through the
example from the docs, which happens to be for disk usage, but i don't see how
i would set up resources such that a request could ask for EITHER disk to have
the required free space and, furthermore, how the job, upon execution, will be
able to determine which disk resource had been allocated it.

my overall aim is to submit about 80,000 jobs in such a way that two of them
remain running, one per disk, untill all jobs are complete.  limiting the
number jobs running is pretty straightforward - the disk situation seems
tougher...  basically i need to manage resources in such a way that every
other job spawned on a node will be running on flip-flopping disk (a-b-a-b...)
and, if one disk fills before the other, simply continue on the other disk
until it's full, when both disk are full the node should become ineligible to
run this particular job.

any ides appreciated.

| EMAIL   :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE   :: 303.497.6469
| ADDRESS :: E/GC2 325 Broadway, Boulder, CO 80305-3328
| URL     :: http://www.ngdc.noaa.gov/stp/
| TRY     :: for l in ruby perl;do $l -e "print \"\x3a\x2d\x29\x0a\"";done 

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list