[GE users] resource management

Ron Chen ron_chen_123 at yahoo.com
Thu Apr 22 03:04:19 BST 2004


Why not create one queue per disk, and each queue is
allowed to run one job.

ie. you will get 2 queues per host:
diskA.q
diskB.q

Then by looking at the $QUEUE environment variable,
the job can figure out which disk is allocated to it.

You can add a load sensor to report the free disk
space for each queue, and even close/disable/suspend
the queue if the free disk space falls below the
requirement.

 -Ron

--- "Ara.T.Howard" <Ara.T.Howard at noaa.gov> wrote:
> sge'rs-
> 
> i have the following situation:
> 
> i am doing some processing where the output needs to
> be distributed evenly to
> two large local disks, call them disk_a and disk_b. 
> each node in the cluster
> has disks by these names of the same capacity.  in
> order for my job to run, it
> must have a certain amount of free space on ONE of
> the two disks, furthermore
> when running it needs knowledge of which disk it was
> allocated (it could
> determine an appropriate disk to run on it's own,
> but not with out race
> conditions i think...).
> 
> i understand generally how resource management works
> and have read through the
> example from the docs, which happens to be for disk
> usage, but i don't see how
> i would set up resources such that a request could
> ask for EITHER disk to have
> the required free space and, furthermore, how the
> job, upon execution, will be
> able to determine which disk resource had been
> allocated it.
> 
> my overall aim is to submit about 80,000 jobs in
> such a way that two of them
> remain running, one per disk, untill all jobs are
> complete.  limiting the
> number jobs running is pretty straightforward - the
> disk situation seems
> tougher...  basically i need to manage resources in
> such a way that every
> other job spawned on a node will be running on
> flip-flopping disk (a-b-a-b...)
> and, if one disk fills before the other, simply
> continue on the other disk
> until it's full, when both disk are full the node
> should become ineligible to
> run this particular job.
> 
> any ides appreciated.
> 
> -a
> -- 
>
===============================================================================
> | EMAIL   :: Ara [dot] T [dot] Howard [at] noaa
> [dot] gov
> | PHONE   :: 303.497.6469
> | ADDRESS :: E/GC2 325 Broadway, Boulder, CO
> 80305-3328
> | URL     :: http://www.ngdc.noaa.gov/stp/
> | TRY     :: for l in ruby perl;do $l -e "print
> \"\x3a\x2d\x29\x0a\"";done 
>
===============================================================================
> 
> 
> 
>
---------------------------------------------------------------------
> To unsubscribe, e-mail:
> users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail:
> users-help at gridengine.sunsource.net
> 



	
		
__________________________________
Do you Yahoo!?
Yahoo! Photos: High-quality 4x6 digital prints for 25?
http://photos.yahoo.com/ph/print_splash

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list