[GE users] qsub -wd usage

Reuti reuti at staff.uni-marburg.de
Sun Oct 26 13:46:34 GMT 2008


Am 26.10.2008 um 02:31 schrieb Bart Willems:

> Hi Reuti,
>
> thanks, this did the trick! Now I just need to copy any output in  
> $TMPDIR
> back to the directory were the source files were initially. Is  
> there an
> environment variable from which I can read what source directory  
> the user
> has set either with -cwd or -wd?

Mmh, up to now my usual approach was either:

START_DIR=`pwd`
START_DIR=$PWD

at the beginning of the script. But there seems to be an unmentioned  
environment variable $SGE_CWD_PATH in 6.2 which can be used for it  
(I'll file an issue to update the `man qsub`page).

If you change the directory only once, you could also use:

cd -

at the end to switch back to the last one and start from there the  
copy process. Means, successive "cd -" can be used to switch back and  
forth between two directories.

-- Reuti


>
> Thanks,
> Bart
>
>> Hi Bart,
>>
>> when you want to run in a scratch directory on a node, you only need
>> the -wd option to locate the source files. This is the directory to
>> which SGE will "cd" when the job starts (default is every user's
>> home, unless you use the -cwd or -wd option). On the node you can use
>> the $TMPDIR which is already created be SGE and is unique for every
>> job (it's location is defined in the queue configuration, where you
>> can replace /tmp with /scratch for your parth):
>>
>> #!/bin/sh
>> #$ -wd /storage/bart/code/
>> cp helloworld $TMPDIR
>> cd $TMPDIR
>> ./helloworld
>>
>> -- Reuti
>>
>>
>> Am 26.10.2008 um 00:42 schrieb Bart Willems:
>>
>>> Hi All,
>>>
>>> I am trying to make our cluster users make use of local scratch  
>>> space
>>> instead of NFS storage while running their jobs. What I would like
>>> to do
>>> is something along the lines of the following job submission script:
>>>
>>> #!/bin/bash
>>>
>>> # This script should be submitted to the batch scheduler with the  
>>> qsub
>>> # command as "qsub <scriptname>".
>>>
>>> # Lines that begin with "#$" are interpreted by the batch scheduler
>>> (SGE)
>>> # as scheduling commands.
>>>
>>> # Use this shell as the default shell
>>> #$ -S /bin/bash
>>>
>>> # Combine stdout and stderr of job
>>> #$ -j y
>>>
>>> # Specify the requested cpu time
>>> #$ -l h_cpu=00:10:00
>>>
>>> # Define source and working directories
>>> SRCDIR=/storage/bart/code/
>>> WRKDIR=/scratch/`whoami`/test
>>>
>>> # Create working directory
>>> mkdir -p $WRKDIR
>>>
>>> # Change to working directory
>>> cd $WRKDIR
>>>
>>> # Copy needed files to working directory.
>>> cp $SRCDIR/helloworld .
>>>
>>> # run job from working directory
>>> #$ -wd $WRKDIR
>>>
>>> # computational command to run
>>> ./helloworld
>>>
>>>
>>> At the end, I would evidently copy back all files from $WRKDIR to
>>> $SRCDIR,
>>> but I 'm not even getting that far. The two problems I seem to be
>>> facing
>>> are
>>>
>>> (1) the -wd option does not like to be followed by a variable
>>> (2) the -wd option must be followed by a directory that already  
>>> exists
>>>
>>> Eventually, I would like the working directory name WRKDIR to be
>>> based on
>>> the job_id to make sure that it 's unique, so I cannot make the
>>> directories in advance.
>>>
>>> Does anyone have any suggestions on how to tackle this?
>>>
>>> Thanks,
>>> Bart
>>>
>>>
>>> -------------------------------------------------------------------- 
>>> -
>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list