[GE users] scheduling with load averages ?

adary adary at marvell.com
Thu Jun 25 14:50:41 BST 2009


Thanks for all the answers. I guess I lost proportions since I never worked with a grid that has lass than 700 cores.

-----Original Message-----
From: dangruhn [mailto:Dan.Gruhn at groupw.com]
Sent: Thursday, June 25, 2009 4:30 PM
To: users at gridengine.sunsource.net
Subject: Re: [GE users] scheduling with load averages ?

Our customers are usually in the 32 core region.  Off the top of my
head, here is what we get (in various different situations):

Calendar based resource allocation - During the day, people are using
their machines and only a single slot is allocated to the grid.  After
hours, all of the slots of a machine open up.  User can queue up jobs
during the day knowing that they aren't stepping on anything and that a
night things really start cooking.

Multiple user negotiation - Chris covered this quite nicely.

Job dependency resolution - Our modeling software runs multiple jobs
with interdependencies handled by SGE.

Job queueing - When a user has work to be done, a simple script runs
that queues up everything needed. SGE then just handles it without the
user having to do anything more.

Dan

craffi wrote:
> There are a few general reasons for running SGE even on small or even
> singular systems. I've seen it used on single machines quite often.
>
> The biggest one is that it allows multiple users, groups or projects
> to use the same hardware without having to shout over cube walls or
> otherwise negotiate access informally. A generic SGE with a simple
> fairshare-by-user policy works quite well for this.
>
> The second big problem is that without something like SGE you are
> either informally scheduling (can be a waste of time or inefficient if
> different locations and timezones are involved) or you are running
> jobs in first-in-first-out (FIFO) order.
>
> The problem with FIFO is that business, project or scientific
> priorities are not reflected in how resources are allocated. In a FIFO
> world, the person who gets to work earliest or logs in first via SSH
> gets the most resources and quite that person is not working on the
> most important task of the day/week/month.
>
> I guess my shortest reply would be that SGE and similar products can
> be of significant benefit on any computer that is used by more than
> one person.
>
> -Chris
>
>
>
> On Jun 25, 2009, at 9:01 AM, adary wrote:
>
>
>> This might sound like a flame, but I assure you it's not:
>>
>> What is the point of using SGE to manage a cluster of 32 cores?
>>
>> I believe that managing something that small is easier by hand ...
>>
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=203566
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
>

--
Dan Gruhn
Group W Inc.
8315 Lee Hwy, Suite 303
Fairfax, VA, 22031
PH: (703) 752-5831
FX: (703) 752-5851

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=203568

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=203576

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list