[GE users] OpenMPI RLIMIT_MEMLOCK problem

pollinger harald.pollinger at sun.com
Thu Dec 4 09:58:47 GMT 2008


Hi Prentice,

"ulimited" is not supported, please use "infinity" instead.

The configuration parser interprets "ulimited" as "0".

Regards,
Harald

Prentice Bisbal wrote:
> I'm using SGE 6.2 with OpenMPI 1.2.8.
> 
> I just setup OpenMPI tight integration following the instructions here:
> 
> http://www.open-mpi.org/faq/?category=running#run-n1ge-or-sge
> 
> I then defined my exec_params to set H_MEMLOCK=unlimited as described here:
> 
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=72405
> 
> Unfortunately, when I submit an MPI job, I still get MEMLOCK errors:
> 
> libibverbs: Warning: RLIMIT_MEMLOCK is 11162 bytes.
>     This will severely limit memory registrations.
> libibverbs: Warning: RLIMIT_MEMLOCK is 11199 bytes.
>     This will severely limit memory registrations.
> libibverbs: Warning: RLIMIT_MEMLOCK is 11181 bytes.
>     This will severely limit memory registrations.
> 
> To make sure my parameters were took effect, I even stopped and
> restarted sge_execd on every compute node with no luck.
> 
> Any ideas? Relevant configuration information is below. Please note that
> in qconf I've tried
> 
> 1. execd_params H_MEMORYLOCKED=ulimited
> 2. execd_params S_MEMORYLOCKED=ulimited
> 3. execd_params S_MEMORYLOCKED=ulimited H_MEMORYLOCKED=ulimited
> 
> All 3 had the same result.
> 
> Let me know if additional/complete config information would be helpful.
> I don't want to flood the list with unnecessary config information.
> 
> # qconf -sp orte
> pe_name            orte
> slots              512
> user_lists         NONE
> xuser_lists        NONE
> start_proc_args    /bin/true
> stop_proc_args     /bin/true
> allocation_rule    $fill_up
> control_slaves     TRUE
> job_is_first_task  FALSE
> urgency_slots      min
> accounting_summary FALSE
> 
> # qconf -sq all.q  | grep pe_list
> pe_list               make orte
> 
> # qconf -sconf | grep execd_params
> execd_params                 H_MEMORYLOCKED=unlimited
> S_MEMORYLOCKED=unlimited
> 
> $ more xhpl.sh
> #$ -S /bin/bash
> #$ -N xhpl
> #$ -pe orte 512
> #$ -cwd
> #$ -V
> 
> MPI=/usr/local/openmpi/gcc-4.1.2/x86_64/
> PATH=${MPI}/bin:$i{PATH}
> LD_LIBRARY_PATH=${MPI}/lib
> 
> mpirun ./xhpl


-- 
Sun Microsystems GmbH         Harald Pollinger
Dr.-Leo-Ritter-Str. 7         N1 Grid Engine Engineering
D-93049 Regensburg            Phone: +49 (0)941 3075-209  (x60209)
Germany                       Fax: +49 (0)941 3075-222  (x60222)
http://www.sun.com/gridware
mailto:harald.pollinger at sun.com
Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
Amtsgericht Muenchen: HRB 161028
Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
Vorsitzender des Aufsichtsrates: Martin Haering

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=91060

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list