[GE users] Can high priority jobs suspend low priority jobs and use their resources?

reuti reuti at staff.uni-marburg.de
Sun May 2 21:00:50 BST 2010


Am 02.05.2010 um 21:46 schrieb reuti:

> Hi,
>
> Am 29.04.2010 um 15:29 schrieb futurity:
>
>> I'm wondering if high priority jobs can suspend low priority jobs
>> and use their suspended resources?  My tests indicate that this
>> isn't possible.
>>
>> Is the grid engine scheduler smart enough to know that a high
>> priority job will suspend a low priority job before it it run on the
>> same machine and therefore calculates the resources that will be
>> available once the low priority job is suspended?
>>
>> If so, is there a way to configuring the grid engine so that high
>> priority jobs are transferred to machines not only when the
>> resources are free, but when there will only be sufficient resources
>> once the subordinate job is suspended?
>>
>> For example, if I have a machine with a consumable "virtual_free"
>> memory of 3000MB.  If a low priority job runs on this machine
>> requesting 1000MB, this reduces the virtual_free to 2000MB.
>>
>> Can a high priority job (one that that would suspend the low
>> priority job) be able to be run on this same machine if it required
>> 2500MB virtual_free memory?  There isn't 2500MB available, only
>> 2000MB so I assume it would be prevented from running on this  
>> machine?
>>
>> In reality, surely if this high job was run on this machine, 500MB
>> of the low priority job's 1000MB memory would be swapped out to disk
>> and the high priority job run in the 2500MB it requires.  Then when
>> the high job is finished, the low priority job would be swapped back
>> to real memory and continued to run.
>>
>> As you can see, its frustrating that high priority jobs are
>> prevented from running on a machine because a resource isn't already
>> available, although it will be available once the low priority job
>> already running there is suspended.
>
> when you have enough virtual_free defined, you could allow a one-time
> swap of the suspended process out of the real memory.

To elaborate this in more detail:

Instead of defining the real memory per exechost (as you just do I  
assume): define an arbitrary high value in the exechost definition  
(hence the load value will always be the limit, but we need something  
to subtract from). Then use an RQS per queue, to limit the real  
consumption again:

limit queues high.q hosts{*} to virtual_free=3000M
limit queues low.q hosts{*} to virtual_free=3000M

-- Reuti


>
> -- Reuti
>
>
>> Kind Regards
>>
>> Neil
>>
>>
>>
>>
>>
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=255839
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net 
> ].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=255840

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list