[GE users] GE6.2 num_proc consumable strange behavior for mpi $fillup jobs

reuti reuti at staff.uni-marburg.de
Mon Feb 2 15:48:20 GMT 2009

Am 02.02.2009 um 16:11 schrieb jlopez:

>> Don't request num_proc here:
>> $ qsub -w v -l num_proc=4,s_vmem=512M,h_fsize=1G,s_rt=300 -pe mpi 4 -
>> q 'sistemas at dn001' test.sh
>> What behavior do you expect? You request 4 slots and there are 4  
>> slots.
>>> But node dn001 has only num_proc=4.
>> You mean you expect num_proc to be multiplied by 4?
> Sorry, I think I did not explain correctly what I want: I want to  
> limit
> the total number of processors used by all the jobs running in a given
> node to the number of processors in that given node.

I can only repeat myself: this can be achieved by the "slots"  
attribut and limit it either on an exechost level (qconf -me <node>)  
or by a resource quota which can be automatically adjust to $num_proc  
(which is a fixed value), as you cite already.

> In the example above I have a node with 4 processors so if a job
> requesting "num_proc=4 and "-pe mpi 4" will consume 4x4=16 processors.
> This would be an example of an hybrid mpi+openmp job.

Aha, so the num_proc was multplied by the requested slot count as  
it's usual done for consumable reqource requests. Why not requesting:

-pe mpi 16

and using a fixed "allocation_rule 4". In you jobscript you will get  
the number of slots ($NSLOST) and hosts ($NHOSTS). Then you can set:

export omp_num_proc=$(($NSLOST/$NHOSTS))

How did you set the omp_num_proc before?

-- Reuti

> In previous versions of GE we achieved this behavior defining num_proc
> as a consumable.
> Cheers,
> Javier
>> -- Reuti
>>> This problem is only happening in GE6.2, our configuration is
>>> working in
>>> previous versions of GE.
>>> Cheers,
>>> Javier
>>> ------------------------------------------------------
>>> http://gridengine.sunsource.net/ds/viewMessage.do?
>>> dsForumId=38&dsMessageId=101302
>>> To unsubscribe from this discussion, e-mail: [users-
>>> unsubscribe at gridengine.sunsource.net].<jlopez.vcf>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do? 
>> dsForumId=38&dsMessageId=101323
>> To unsubscribe from this discussion, e-mail: [users- 
>> unsubscribe at gridengine.sunsource.net].
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=101374
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].<jlopez.vcf>


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list