[GE users] Need help in SGE configuration and resource allocation

cjf001 john.foley at motorola.com
Sun May 2 02:21:26 BST 2010


Shaila -

here's how we handle a similar situation - using your example,
we would have 3 hostgroups, as you described them. Also, we
have 2 cluster queues - called "primary" and "secondary". Those
in Mr A's group have access to the primary queue on hostgroup 2,
and those in Mr B's group have access to the primary queue
on hostgroup 3. If hostgroup 1 is truly shared, and has no
users that have "priority", then in our model we would not give
anyone access to the primary queue on hostgroup 1. All users
have access to the secondary queue on all hostgroups.

Now, the thing that makes this work is that the secondary queue
is a subordinant of the primary queue. Therefore, for example,
if a user who's not in either hostgroup 2 or hostgroup 3 starts
a job, it might get assigned to the secondary queue on hostgroup 2.
That's OK as long as no one in Mr A's group needs it. But if
someone in Mr A's group does submit a job, and there's nowhere
else to run it, then the secondary job on hostgroup 2 will get
suspended, and Mr A's guy's job will run there.

That's how we handle resources that are "paid for" by a certain
group of users, and expect to have access to them, while sharing
them with all users when they aren't busy. If that sounds like
something you'd want to pursue, I can send more details, but that's
enough for this post....

     John


shaila wrote:
> Hi
>
> I am a newbie to the resource allocation and ticket allocation on SGE.
> I have the following scenario :-
>
> We have 3 sets of machines :-
>
> HostGroup 1 - 7 machines with 8 cores each and 16GB of RAM
> HostGroup 2 - 4 machines with 16 cores  and 24G of RAM
> HostGroup 3 - 10 machines with 8 cores and 24GB of RAM
>
> Hostgroup 1 has been purchased by the college, Hostgroup 2 has been purchased by Mr. A and Hostgroup 3 has been purchased by Mr. B. They have included them in the big cluster with the condition that when they submit a job, they will get a minimum of what they purchased and additional resources from the other hostgroups based on availability.
> So, when members from Mr. A's group  submit a job, they should get at least 64 cores and 24x4 GB of RAM and if there is more available , then they should get that too.
> But when someone from Mr. B's group  comes along and submits a job, he should be guaranteed the 80 cores and 24x10 GB of RAM. So, if there is another job running on the other nodes, then they should be suspended and his jobs should start.
> We also need to have a maximum wait period of 4 hours for the jobs.
>
> I feel that there are a lot of things to be considered here and am not sure how to go about it.
>
> I know I will need to have a load_scaling factor. But I am not able to figure out how to calculate it.
> Also , I since we do not need to have a share tree policy, I have configured a functional policy for the groups.
>
> Any suggestions on how to start on this sort of resource allocation and scheduling would be extremely helpful.
>
> Thanks a lot in advance.
>
> Shaila
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=255609
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



-- 
###########################################################################
# John Foley                          # Location:  IL93-E1-21S            #
# IT & Systems Administration         # Maildrop:  IL93-E1-35O            #
# Antenna & Mechanical Simulation Grp #    Email: john.foley at motorola.com #
# Motorola, Inc. -  Mobile Devices    #    Phone: (847) 523-8719          #
# 600 North US Highway 45             #      Fax: (847) 523-5767          #
# Libertyville, IL. 60048  (USA)      #     Cell: (847) 460-8719          #
###########################################################################
               (this email sent using SeaMonkey on Windows)

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=255748

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list