[GE users] qrsh consumes consumables, qsub does not

Ondrej Bojar bojar at ufal.mff.cuni.cz
Fri Nov 21 23:36:28 GMT 2008


Hi,

a few more observations until my previous mail arrives.

I can reproduce the described behaviour even on a busy cluster, all I have to do 
is use '-now no' for qrsh.

The behaviour is the same for mem_free and virtual_free. By a lucky coincidence, 
both the qrsh and the qsub attempts got scheduled to the same execution host. I 
used '-l virtual_free=10G' for both qrsh and qsub. I used 'qhost -F' to check 
the particular host.

With neither qsubmitted nor qrshed job, qhost -F shows:
    hc:virtual_free=31.000G
    (corresponds to our setting for the host)

With qsubmitted job running, the value remains unchanged.

With qrshed job running, the value decreases as expected and qhost shows:
    hc:virtual_free=21.000G


I confirmed exactly the same for mem_free. The experiment for mem_free happened 
on another execution host, but the prefix 'hc:' for mem_free was there. So the 
resource appears to be correctly recognized as a consumable.

Any ideas?

Thanks, Ondrej.


reuti wrote:
> Hi,
> 
> Am 21.11.2008 um 15:51 schrieb Ondrej Bojar:
> 
> 
>>Dear all,
>>
>>we use GE 6.1u3 and we have set mem_free to consumable:
>>
>>qconf -sc says:
>>...
>>mem_free   mf   MEMORY    <=   YES    YES        100      0
>>...
>>
>>All our execution hosts have mem_free set to their available physical
>>memory in qmon->Host Configuration->Consumables/Fixed Attributes.
>>
>>I can list hosts satisfying some minimum free memory limit:
>>
>>    qhost -l mem_free=15G
>>
>>
>>I can schedule interactive jobs requiring (reserving) some amount of
>>this mem_free consumable resource:
>>
>>    qrsh -l mem_free=15G "hostname; sleep 60"
>>
>>Checking the list of hosts (qhost -l above) confirms that the resource
>>has been partially consumed, e.g. the used execution host disappears
>>from the list.
> 
> 
> this I don't see.
> 
> 
> 
>>Submitting a job with 'qsub -l mem_free=15G ...' however submits  
>>the job
>>on any free execution host, regardless of mem_free. Moreover, the  
>>'qhost
>>-l' list remains unchanged.)
>>
>>Could you think of any explanation?
>>
>>
>>A side issue is that even the 'qrsh -l mem_free=15G' is not  
>>reliable. I
>>often get the error 'Your "qrsh" request could not be scheduled, try
>>again later.', even in case there are enough hosts available in the
>>'qhost -l ...' listing.
>>
>>
>>(Our motivation is clear, we want to trust users: if someone submits a
>>job claiming he'll need 15 GB, we don't want to another 15GB job on a
>>16GB machine, even in case the job in question is not consuming its 15
>>GB yet.)
> 
> 
> a) Do you also observe, that the "mem_free" output is always prefixed  
> by a "hl:" in `qhost -F`?
> 
> b) Can you try the same with virtual free? - You should see a "hc:"  
> in front of it then.
> 
> As it's nowhere mentioned to be a feature of virtual_free only, maybe  
> it's a side effect of mem_tot/mem_used being displayed as columns in  
> the usual qhost output also. When you define it also as a consumable:  
> shall these columns also change their output?
> 
> -- Reuti
> 
> 
> 
>>Looking forward to any suggestions,
>>   Ondrej Bojar.
>>
>>-- 
>>Ondrej Bojar (mailto:obo at cuni.cz / bojar at ufal.mff.cuni.cz)
>>http://www.cuni.cz/~obo
>>
>>------------------------------------------------------
>>http://gridengine.sunsource.net/ds/viewMessage.do? 
>>dsForumId=38&dsMessageId=89374
>>
>>To unsubscribe from this discussion, e-mail: [users- 
>>unsubscribe at gridengine.sunsource.net].
> 
> 
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=89438
> 
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

-- 
Ondrej Bojar (mailto:obo at cuni.cz / bojar at ufal.mff.cuni.cz)
http://www.cuni.cz/~obo

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=89462

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list