[GE users] Array jobs have priority?

Terri Kamm tkamm at acm.org
Fri Mar 30 14:35:24 BST 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

I may have found another fix. I changed "maximum pending tasks per
job" from 50 to 1. This seems to have enabled some turn taking between
the two users on the same project. However, while watching it the last
half hour, more than 200 jobs from the array have been scheduled while
user 2 has 3 jobs waiting in the queue.

The behavior that I saw before making the change was that if N(<50)
slots were free and the array job had this highest priority over all
users, it scheduled N jobs from the array job. Now it is scheduling
less than N tasks from the array job, but still more than one.

What I still don't understand is how the array job kept getting the
higher priority over user 2. User 1's priority should be continually
decreasing because he has used resources recently. Is it not getting
penalized for the compute already used for the first tasks? Is the
usage accumulated "per job" rather than per task, so it won't have an
impact until after the whole array jobs completes?

-- 
Terri Kamm
tkamm at acm.org

On 3/30/07, Daniel Templeton <Dan.Templeton at sun.com> wrote:
> Terri,
>
> I missed the "array job" part in my previous answer.  To answer the
> specific question you asked, an array job is treated as a single job for
> the purposes of tickets.  Every task of the array job effectively gets
> the same number of tickets.  (Someone correct me if I'm wrong, please.)
> That means that the functional ticket policy would have the same
> problem.  I think the answer for you is specifically to set up a wait
> time policy.  That way, if a priority user blocks the system with a very
> large array job, other users' jobs will still have a chance to run.
>
> Daniel
>
> Daniel Templeton wrote:
> > Terri,
> >
> > The sharetree ticket policy assigns shares to jobs according to the
> > sharetree and the order of the jobs.  Unless you've assigned override
> > tickets, the order of the jobs when the sharetree is evaluated will be
> > the submit order.  That means that if both users are in the same
> > project and have the same share entitlement, the sharetree tickets
> > will just enforce the submit order with respect to those two users' jobs.
> >
> > There are several ways to deal with the problem.  For a one-off fix,
> > just assign the second user enough override tickets to get his jobs
> > scheduled.  For a real fix, you should probably configure functional
> > tickets and/or urgency tickets.  Alternatively, if it makes sense, you
> > could change the sharetree entitlement for one or both users.
> >
> > Daniel
> >
> > Terri Kamm wrote:
> >> I have a scheduling issue that I don't understand.
> >>
> >> I have project-based share tree scheduling based on 100%CPU and an 8
> >> hour halflife. I have two users who are on the same project. One user
> >> submitted a large array job yesterday with 33000 jobs. So far 20000
> >> have run, so he has has used a lot of the resources. I have a second
> >> user on the same project who has submitted a few jobs today. These
> >> jobs are not ever sorted with a higher priority than the first user's
> >> array job, so the second user's jobs are not begin run. In fact, user
> >> one's array jobs are being assigned 10-20 at a time while user two
> >> continues to wait.
> >>
> >> I understand how the share tree scheduling works across projects. I
> >> thought that within the project the users would be handled
> >> "round-robin" or some sort of fair share as well. What am I not
> >> understanding?
> >>
> >> --
> >> Terri Kamm
> >> tkamm at acm.org
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list