[GE users] How to move resultfiles from execute host back to submitter

Atle Rudshaug atle.rudshaug at gmail.com
Fri Dec 14 18:21:07 GMT 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

My setup is as follows:
Node 1: Submit/execute
Node 2: Submit/execute
Node 3: Master/Submit/execute

I only get this error on Node 3, which is the one I am sending jobs
from and results back to. I guess SCP does not like transferring files
to itself. Guess I need an if to avoid sending results from Node 3 to
Node 3...

- Atle

On 12/14/07, Daniel Templeton <Dan.Templeton at sun.com> wrote:
> Atle,
>
>  From what machine are you submitting the job?  I'm really confused as
> to how the SSH login window has any idea *where* to pop up.
>
> Just to double-check, I set up an epilog script that scp's a file across
> the network.  When I submit a job with my id in the authorized_keys
> file, everything works fine.  When I submit a job without my id in the
> authorized_keys file, I get the following line in the job's error file:
>
> Permission denied
> (gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive).
>
> and qacct says the job failed in the epilog, which is also correct behavior.
>
> So, the question is, what in your environment is causing the odd
> behavior?  What happens if you leave out the epilog and submit a job
> that tries to do an scp?
>
> Daniel
>
> Atle Rudshaug wrote:
> > It works just fine when executing the script by hand on the execution
> > hosts. Everything is transferred without problems. However, when
> > submitting a job from a terminal window inside Gnome X, an OpenSSH
> > dialog appears and wants me to enter my password for each file
> > transfer. However, if submitting from command promt outside X (Ctrl +
> > Alt + F1), the files are transferres successfully without promting for
> > password as I want it to.
> >
> > - Atle
> >
> > On 12/14/07, Daniel Templeton <Dan.Templeton at sun.com> wrote:
> >
> >> The epilog script is just a script.  There shouldn't be any strange
> >> behavior.  It's executed as the user who submitted the job on the host
> >> where the job will be run.  What happens when you run the script by hand
> >> on the execution host?
> >>
> >> Daniel
> >>
> >> Atle Rudshaug wrote:
> >>
> >>> Well the epilog script starts. I had forgotten to make it executable.
> >>> Now I get another problem with SSH. An OpenSSH dialog appears on the
> >>> submit host and asks for my password for each file. An error dialog in
> >>> the background says that the SSH password dialog could not grab the
> >>> keyboard input.
> >>>
> >>> scp works fine manually when I run the epilog script from the execute
> >>> hosts. Why do these dialog boxes appear for every file when run it in
> >>> the epilog script? Is this a known problem? RCP just hangs if I try
> >>> that.
> >>>
> >>> - Atle
> >>>
> >>> On 12/14/07, Daniel Templeton <Dan.Templeton at sun.com> wrote:
> >>>
> >>>
> >>>> Harald,
> >>>>
> >>>> Hmmm...  You added that feature, and you wrote the howto page, but I'm
> >>>> going to be bold and disagree with you.  My understanding is that
> >>>> delegated file staging is really just a switch that lets you tell the
> >>>> qmaster that you have configured your own epilog and prolog scripts to
> >>>> enable file staging, so that the DRMAA transfer_files optional attribute
> >>>> is accepted for DRMAA applications.  It that's true, then it doesn't
> >>>> help Atle set up the epilog script in the first place.  Delegated File
> >>>> Staging is the step *after* solving Atle's problem, so that a DRMAA
> >>>> application could take advantage of the solution.
> >>>>
> >>>> Do I have it wrong?
> >>>>
> >>>> Daniel
> >>>>
> >>>> Harald Pollinger wrote:
> >>>>
> >>>>
> >>>>> Hi Atle,
> >>>>>
> >>>>> there is a mechanism called "delegated file staging" in SGE. It's
> >>>>> called "delegated" because it just provides all necessary
> >>>>> informations, the real work is delegated to the prolog/epilog scripts.
> >>>>>
> >>>>> Please see the man page sge_conf(5) how to enable and how to use this
> >>>>> mechanism, search terms in this page are "delegated_file_staging" and
> >>>>> "$fs_", and
> >>>>> http://gridengine.sunsource.net/howto/filestaging/filestaging6.html
> >>>>> for a howto.
> >>>>>
> >>>>> Regards,
> >>>>> Harald
> >>>>>
> >>>>> Atle Rudshaug wrote:
> >>>>>
> >>>>>
> >>>>>> Hi!
> >>>>>>
> >>>>>> I am doing some tests with SGE and Condor on 3 nodes only. What I need
> >>>>>> is to get the result files for SGE back to my home dir on the submit
> >>>>>> host. I don't have the home directories on NFS and don't have time to
> >>>>>> set that up before a deadline. Condor does this automatically. Is
> >>>>>> there some automatic way for SGE? Do I have to use a epilogue script?
> >>>>>> If so, how?
> >>>>>>
> >>>>>> - Atle
> >>>>>>
> >>>>>> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>>>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>>>>>
> >>>>>>
> >>>>>>
> >>>> ---------------------------------------------------------------------
> >>>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>>> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>>>
> >>>>
> >>>>
> >>>>
> >>> ---------------------------------------------------------------------
> >>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>>
> >>>
> >>>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >> For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>
> >>
> >>
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> >
> >
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list