[GE users] h_fsize and mpi

pablorey prey at cesga.es
Wed Dec 16 13:55:48 GMT 2009


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

    Hi,

    You are right, it was set to 200G or 400G as you said. Problems with the copy/paste method.....

    As you said, you have made h_fsize consumable and so SGE reject that job.

    Regards,
    Pablo



On 16/12/2009 13:42, reuti wrote:

Hi,

Am 16.12.2009 um 13:27 schrieb pablorey:



    Hi Reuti,

    I have done the test that you suggested and it seems that it
confirm my thesis.

    I have reserved a node with h_fsize=400G and I done the
following tests (mpi_4p is configure to use 4 slots per node):
    1) qsub h_fsize=100G -pe mpi_4p 4: The job ran in the reserved
nodo and the file size limit was set to 100G.
    2) qsub h_fsize=200G -pe mpi_4p 4: The job ran in other node
with h_fsize=900G and the file size limit was set to 100G.



typo: "... was set to 200G" you meant?



    3) qsub h_fsize=400G -pe mpi_4p 4: The job was not enqueued
because we don't have nodes with h_fsize>=1600G.



Did you make h_fsize consumable? Otherwise SGE shouldn't reject it I
think.




    4) qsub h_fsize=400G -pe mpi 4: The job ran in 3 nodes (2 of
them with h_fsize=400G and other with h_fsize=900G) and the file
size limit was set to 100G.



typo: "... was set to 400G" you meant?

-- Reuti




    So we can deduced that as you said the limit is per process,
and that GE do the same as it is verified in the case 3) and 4).

    Thanks by your suggestion,
    Pablo



On 16/12/2009 11:43, reuti wrote:


Hi, Am 16.12.2009 um 11:09 schrieb pablorey:


Hello, I have a doubt about h_fsize and MPI jobs. In a MPI job,
the h_fsize is limited by slot or limited by node?. I know that
the s_vmen value is the limit for each slot. For example, if a
sent the following job: qsub -l
num_proc=1,s_rt=10:00:00,s_vmem=2G,h_fsize=20G -pe mpi 32 job.sh


like all system limits it's per process. To check it, you submit a
job with a script like: #!/bin/sh ulimit -aH echo ulimit -aS to
get the set limits output. Only limits handled by SGE are
different (man queue_conf), i.e. h/ s_cpu and h/s_vmem. -- Reuti


the memory limit by each slot is established to 2G, so in each
node the memory limit is set to "2G * number_of_assigned_slots".
What about the h_fsize limit?. Is similar to s_vmen or it is set
by node?. That is, in the previous example, the limit is 20G or
"20G * number_of_assigned_slots". Regards. -- Pablo Rey Mayo
Tecnico de Sistemas Centro de Supercomputacion de Galicia (CESGA)
Avda. de Vigo s/n (Campus Sur) 15705 Santiago de Compostela
(Spain) Tel: +34 981 56 98 10 ext. 233; Fax: +34 981 59 46 16
email: prey at cesga.es<mailto:prey at cesga.es>; http://www.cesga.es/
------------------------------------------------ NOTA: Este
mensaje ha sido redactado intencionadamente sin utilizar acentos
ni caracteres especiales, para que pueda ser visualizado
correctamente desde cualquier cliente de correo y sistema.
------------------------------------------------ <xacobeo.jpg>


------------------------------------------------------ http://
gridengine.sunsource.net/ds/viewMessage.do?
dsForumId=38&dsMessageId=233677 To unsubscribe from this
discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net<mailto:users-unsubscribe at gridengine.sunsource.net>].


--
Pablo Rey Mayo
Tecnico de Sistemas
Centro de Supercomputacion de Galicia (CESGA)
Avda. de Vigo s/n (Campus Sur)
15705 Santiago de Compostela (Spain)
Tel: +34 981 56 98 10 ext. 233; Fax: +34 981 59 46 16
email: prey at cesga.es<mailto:prey at cesga.es>; http://www.cesga.es/
------------------------------------------------
NOTA: Este mensaje ha sido redactado intencionadamente sin utilizar
acentos ni caracteres especiales, para que pueda ser visualizado
correctamente desde cualquier cliente de correo y sistema.
------------------------------------------------
<xacobeo.jpg>



------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=233698

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net<mailto:users-unsubscribe at gridengine.sunsource.net>].




--
Pablo Rey Mayo
Tecnico de Sistemas
Centro de Supercomputacion de Galicia (CESGA)
Avda. de Vigo s/n (Campus Sur)
15705 Santiago de Compostela (Spain)
Tel: +34 981 56 98 10 ext. 233; Fax: +34 981 59 46 16
email: prey at cesga.es<mailto:prey at cesga.es>; http://www.cesga.es/
------------------------------------------------
NOTA: Este mensaje ha sido redactado intencionadamente sin utilizar
acentos ni caracteres especiales, para que pueda ser visualizado
correctamente desde cualquier cliente de correo y sistema.
------------------------------------------------

[cid:part1.02090807.00070103 at cesga.es]


    [ Part 2, "xacobeo.jpg"  Image/JPEG (Name: "xacobeo.jpg") 28 KB. ]
    [ Unable to print this part. ]



More information about the gridengine-users mailing list