[GE users] GE6.2 num_proc consumable strange behavior for mpi $fillup jobs

reuti reuti at staff.uni-marburg.de
Mon Feb 2 15:48:20 GMT 2009


Am 02.02.2009 um 16:11 schrieb jlopez:

>> Don't request num_proc here:
>>
>> $ qsub -w v -l num_proc=4,s_vmem=512M,h_fsize=1G,s_rt=300 -pe mpi 4 -
>> q 'sistemas at dn001' test.sh
>>
>> What behavior do you expect? You request 4 slots and there are 4  
>> slots.
>>
>>
>>> But node dn001 has only num_proc=4.
>>>
>>
>> You mean you expect num_proc to be multiplied by 4?
>>
> Sorry, I think I did not explain correctly what I want: I want to  
> limit
> the total number of processors used by all the jobs running in a given
> node to the number of processors in that given node.

I can only repeat myself: this can be achieved by the "slots"  
attribut and limit it either on an exechost level (qconf -me <node>)  
or by a resource quota which can be automatically adjust to $num_proc  
(which is a fixed value), as you cite already.

> In the example above I have a node with 4 processors so if a job
> requesting "num_proc=4 and "-pe mpi 4" will consume 4x4=16 processors.
> This would be an example of an hybrid mpi+openmp job.

Aha, so the num_proc was multplied by the requested slot count as  
it's usual done for consumable reqource requests. Why not requesting:

-pe mpi 16

and using a fixed "allocation_rule 4". In you jobscript you will get  
the number of slots ($NSLOST) and hosts ($NHOSTS). Then you can set:

export omp_num_proc=$(($NSLOST/$NHOSTS))

How did you set the omp_num_proc before?

-- Reuti


>
> In previous versions of GE we achieved this behavior defining num_proc
> as a consumable.
>
> Cheers,
> Javier
>> -- Reuti
>>
>>
>>
>>> This problem is only happening in GE6.2, our configuration is
>>> working in
>>> previous versions of GE.
>>>
>>> Cheers,
>>> Javier
>>>
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?
>>> dsForumId=38&dsMessageId=101302
>>>
>>> To unsubscribe from this discussion, e-mail: [users-
>>> unsubscribe at gridengine.sunsource.net].<jlopez.vcf>
>>>
>>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do? 
>> dsForumId=38&dsMessageId=101323
>>
>> To unsubscribe from this discussion, e-mail: [users- 
>> unsubscribe at gridengine.sunsource.net].
>>
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=101374
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].<jlopez.vcf>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=101392

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list