[GE users] File staging problem

Pursley, Kevin J kevin.pursley at bp.com
Tue Jul 8 12:54:01 BST 2008


This is an epilog that we use to clean scratch space on all nodes after
an MPI job.
We use loose integration so SGE just aloocates slots for the machines
and starts
A single process on the master node. Don't know about tight mpi
integration, we don't use that.
Also note that prologs and epilogs run as the user, nothing special
there either. 


#!/bin/csh -f
# clean /scratch
echo " start woodcrest epilog "
foreach i (`cat $TMPDIR/machines`)
   /usr/bin/rsh $i 'rm -Rf /scratch/*'
End
echo " end woodcrest epilog " 

-----Original Message-----
From: Gerhard Venter [mailto:gventer at sun.ac.za] 
Sent: Tuesday, July 08, 2008 2:33 AM
To: users at gridengine.sunsource.net
Subject: [GE users] File staging problem

I am using SGE 6.0 and OpenMPI.  I am distributing my job to multiple
slots using a round robin mpi parallel environment - the result is that
each slot is on a different compute node, each with its own local disk
storage.  I would like to stage files from my home directory to the
TMPDIR on each compute node that MPI will use.  My submit script looks
something like this:

#!/bin/bash
#$ -cwd
#$ -j y
#$ -pe openmpi_rr 4
#
cp input.dat $TMPDIR
cd $TMPDIR
mpirun -np ${NSLOTS} $SGE_O_WORKDIR/mpijob

The cd $TMPDIR does change the pwd on each node to $TMPDIR (I print the
value for all slots from my mpi program), however, the files are only
copied to the first MPI slot (the master slot) and not the other slots
(the worker slots).  I have also implemented the prolog and epilog
scripts from

http://gridengine.sunsource.net/project/gridengine/howto/filestaging/

but with the same result.  

Am I missing something?  Is there a way that I can stage input files for
all the slots that will be used in a MPI job?

Thanks,
Gerhard

-- 
+------------------------------------------------------------------+
|| Prof. Gerhard Venter
||
|| Departement Meganiese en        |  Department of Mechanical and
||   Megatroniese Ingenieurswese   |    Mechatronic Engineering
|| Universiteit Stellenbosch       |  Stellenbosch  University
|| Privaat Sak X1 Matieland 7602   |  Private Bag X1 Matieland 7602  
|| Suid-Afrika                     |  South Africa
||
|| Tel: +27 21 808 3560
|| E-Mail: gventer at sun.ac.za          Web: www.eng.sun.ac.za
+------------------------------------------------------------------+


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list