[GE users] Small jobs jumping ahead of large jobs that reserve Virtual_free memory.

reuti reuti at staff.uni-marburg.de
Fri Feb 6 12:01:35 GMT 2009


    [ The following text is in the "UTF-8" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hi,

Am 05.02.2009 um 19:07 schrieb futurity:

> The admin manual talks about the jobs with the highest priority get  
> the earliest resource assignment.
>
> I have a few questions about this. When we submit jobs (all with  
> the same priority) using up a consumable virtual_free ?vf? memory  
> resource, we see in a queue with a small jobs running on every  
> machine, that small jobs that are requesting a small amount of ?vf?  
> but aren?t reserving ?vf? are transferring and running ahead of  
> large jobs that are requesting a large amount of ?vf? and are  
> reserving ?vf?.  These smaller jobs are submitted after the larger  
> jobs are submitted.

what SGE version are you using? Maybe you see the "backfilling", as  
no h_rt was requested and the default_duration is like in older SGE  
versions still set to 10 minutes - although the small jobs will need  
more than 10 minutes in the end. SGE might put them on nodes, which  
are reservered in 12 minutes, but then the node is still blocked by  
the smaller job.

Do all your jobs request h_rt?

-- Reuti

>
> 1a)   Does resource assignment and reservation work when jobs have  
> equal priority?
> 1b)   In the example scenario described above, will the large jobs  
> always be held back as long as there are smaller jobs waiting to be  
> run (queued before it and after it)?
> 1c)   We don't want the users submitting these larger to have an  
> unfair advantage in the grid, so we ideally don?t want them to be  
> able to submit jobs of a higher priority.  Do these larger jobs  
> have to have a higher priority?  If they have to have higher  
> priority, is there a way to stop the users abusing this priority  
> advantage without stopping them from using all machines when the  
> grid is empty.
>
> I?ve tried to test various scenarios to replicate this problem, but  
> its a nightmare tracking what is going on when I?m submitting jobs  
> to more than one machine at a time.
>
> Any help will be really appreciated.
>
> Regards
>
> Neil

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=102382

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list