[GE users] File copy from local /scratch on job termination

Bart Willems b-willems at northwestern.edu
Mon Dec 15 14:09:18 GMT 2008

Hi All,

I recently urged our cluster users to use local scratch space on the
cluster nodes instead of the NFS mounted RAID during their calculations.
In the example submission file below all required files for the job are
copied over to the node's local hard disk ($TMPDIR is /scratch) and copied
back when the job completes. However, the files only get copied back when
the job exits normally. If SGE terminates the job because it exceeds the
requested CPU time or if a user manually terminates a job with qdel, the
files are not copied back from the node's local hard disk to the RAID. Is
there any way around this?



#$ -S /bin/bash
#$ -j y
#$ -N helloworld_test
#$ -l h_cpu=00:02:00
#$ -cwd

# Copy job files to local scratch space
tar cfz $JOBFILE helloworld
rm -rf $JOBFILE
tar xfz $JOBFILE
rm -rf $JOBFILE

# Computational command to run

# Copy all files back.
tar cfz $OUTFILE *
tar xfz $OUTFILE
rm -rf $OUTFILE


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list