[GE users] How to move resultfiles from execute host back to submitter

Atle Rudshaug atle.rudshaug at gmail.com
Fri Dec 14 16:46:42 GMT 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

I can't get it to work. I use paswordless ssh-keys between the hosts
and scp works just fine.

1. I copied the epilog script from your tutorial and rewrote it to:

for file in $(ls dagoc.sh.*) ;
do
scp /home/atle/$file $SGE_O_HOST:/home/atle/
done

2. I put this in a file in the path for epilog in the all.q and the
global cluster config.

3. I submit a job:
qsub -t 1-10 job.sh

4. When running 'qacct' on my job, I get:
failed 14 : before epilog

Are there any special flags i need to set? I just want some numbers
sent back to the submit host from the distributed jobs.

- Atle



On 12/14/07, Harald Pollinger <Harald.Pollinger at sun.com> wrote:
> Atle Rudshaug wrote:
> > Hmm...It seems that file staging is only possible with DRMAA. Is that
> > true?
>
> No.
>
>
> > In my project I don't use DRMAA. I need the epilog script with
> > the file transfer to happen after an array job is done.
>
> Just do it like in the howto. It tells you how to configure SGE and what
> to add to your prolog and epilog scripts.
>
> Regards,
> Harald
>
>
> > Is there not
> > any other way than use DRMAA? Could I just manually write a regular
> > job submit file that uses SCP and transfers all output files on the
> > execution host back to the submitter? Then send this script as
> > dependent on the array job?
> >
> > - Atle
> >
> >
> > On 12/14/07, Harald Pollinger <Harald.Pollinger at sun.com> wrote:
> >> Hi Atle,
> >>
> >> there is a mechanism called "delegated file staging" in SGE. It's called
> >> "delegated" because it just provides all necessary informations, the
> >> real work is delegated to the prolog/epilog scripts.
> >>
> >> Please see the man page sge_conf(5) how to enable and how to use this
> >> mechanism, search terms in this page are "delegated_file_staging" and
> >> "$fs_", and
> >> http://gridengine.sunsource.net/howto/filestaging/filestaging6.html for
> >> a howto.
> >>
> >> Regards,
> >> Harald
> >>
> >> Atle Rudshaug wrote:
> >>> Hi!
> >>>
> >>> I am doing some tests with SGE and Condor on 3 nodes only. What I need
> >>> is to get the result files for SGE back to my home dir on the submit
> >>> host. I don't have the home directories on NFS and don't have time to
> >>> set that up before a deadline. Condor does this automatically. Is
> >>> there some automatic way for SGE? Do I have to use a epilogue script?
> >>> If so, how?
> >>>
> >>> - Atle
>
> --
> Sun Microsystems GmbH         Harald Pollinger
> Dr.-Leo-Ritter-Str. 7         N1 Grid Engine Engineering
> D-93049 Regensburg            Phone: +49 (0)941 3075-209  (x60209)
> Germany                       Fax: +49 (0)941 3075-222  (x60222)
> http://www.sun.com/gridware
> mailto:harald.pollinger at sun.com
> Sitz der Gesellschaft: Sonnenallee 1, D-85551 Kirchheim-Heimstetten
> Amtsgericht Muenchen: HRB 161028
> Geschaeftsfuehrer: Thomas Schroeder, Wolfgang Engels, Dr. Roland Boemer
> Vorsitzender des Aufsichtsrates: Martin Haering
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list