[GE users] Gridengine and Abaqus HP-MPI parallel computing integration

Gautier, Valentin Valentin.Gautier at eads.net
Thu Jun 22 15:51:17 BST 2006


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

 

 

Thanks, for the example, it's seems indeed pretty tricky.

 

But how can I get the machines file from sge ?

 If I could , I think I would generate a dynamic abaqus_v6.env with the adequate mp_host_list.

 

 

 

________________________________

De : Jeroen M. Kleijer [mailto:jeroen.m.kleijer at philips.com] 
Envoyé : jeudi 22 juin 2006 15:31
? : users at gridengine.sunsource.net
Objet : Re: [GE users] Gridengine and Abaqus HP-MPI parallel computing integration

 


Hi, 

It's been over a year since I've looked into it. 
I had it working but since most of our jobs didn't benefit from adding cpus we left it at that. 
There are a few pointers: 
- submission goes through the abaqus_v6.env file but I think you already figured that one out yourself 
  here's a snippet from the one we use: 

        mp_mode = MPI 
        mp_file_system = (DETECT,DETECT) 
        mp_file_system = (SHARED,LOCAL) 
        mp_mpi_implementation = MPICH 
        mp_mpirun_path = {LAM:'/cadappl/lam/7.0.4-64.sge/bin/mpiexec-abaqus', 
                          MPICH:'/cadappl/mpich/1.2.7p1-64.sge/bin/mpirun'} 
        
        ##mp_mpirun_options = " -ssi rpi usysv -ger -O " 
        ##mp_mpirun_options = " -ssi rpi usysv -ger -O " 
        ##mp_rsh_command = 'rsh -n -l %U %H "export PATH=/cadappl/lam/7.0.4-64.sge/bin:${PATH};%C"' 
        mp_rsh_command = 'rsh -n -l %U %H %C' 
        run_mode = BATCH 
        from driverQueues import * 
        class SGE_Queue(LSFQueue): 
            def __repr__(self): 
                return 'Submits to SGE %s queue ' 
            def submit(self, options, env): 
                job = options['job'] 
                verbose = options.get('verbose', 0) 
                queue = self.name 
                cpus = options.get('cpus','1') 
                ##qsub = '/home/sge/bin/lx24-amd64/qsub -pe mpich %s -V -v LAMHOME -v PATH -l arch=lx24-amd64 -cwd -q %s -N %s -o %s.log -b y                 "~/.caddata/abaqus/abaqus python %s.com"' % \ 
                qsub = '/home/sge/bin/lx24-amd64/qsub -V -v LAMHOME -v PATH -l arch=lx24-amd64 -cwd -q %s -N %s -o %s.log -b y                                 "~/.caddata/abaqus/abaqus python %s.com"' % \ 
                       (queue, job, job, job) 
        ##               (cpus, queue, job, job, job) 
                return self.spawn(qsub, env, verbose) 
        ##queues['batch.q']   = SGE_Queue(name='batch.q') 
        queues['default']   = SGE_Queue(name='batch.q') 
        queues['immediate.q']  = SGE_Queue(name='immediate.q') 
        queues['hold.q']  = HoldQueue() 
- as you can see we switched back to single cpu  because of several issues we ran in to 
- when you submit the job (an .inp file) abaqus check whether there's a file called abaqus_v6.env (in either your job or your homedirectory) and checks whether you've got a variable called mp_host_list with the exact same number of cpus you requested. (so if you've requested 3 cpus, it should hold something like [['machine1',1],['machine3',2]] Whether these machines actually exist doesn't matter, the line should exist! 
- the problem you then get is getting the machines file, created by SGE to be used by Abaqus which is a different matter. 
- also bear in mind that the syntax for the machines file differs per implementation.... mpich wants it another way than HP-MPI 

I hope this gives you some idea on where to look but it's been a while since I looked into it because of the major headache it can turn into. 

Met vriendelijke groeten / Kind regards

Jeroen Kleijer
Unix Systeembeheer
Philips Applied Technologies 








"Gautier, Valentin" <Valentin.Gautier at eads.net> 

2006-06-22 03:01 PM 

Please respond to
users at gridengine.sunsource.net

To

<users at gridengine.sunsource.net> 

cc

 

Subject

[GE users] Gridengine and Abaqus HP-MPI parallel computing         integration 

Classification

 

 

 

 




Hello everybody, 
  
We've got a 6 nodes bi-opteron linux cluster used for abaqus computing,  I've got  no problem to get abaqus work with  SGE  in threads multi-processing mode, but I have no idea of how to make it work in mpi mode with hp-mpi ( delivered with abaqus 6.6-1). 
Does anyone have some sort of solution to configure the Gridengine mpi queue ? 
Is there any howto for that ? 
  
Thanks a lot. 
  
  



    [ Part 2: "Attached Text" ]

    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net



More information about the gridengine-users mailing list