[GE users] Delegated filestaging problem

sumyee sum-yee.lai at hp.com
Wed Mar 17 19:10:04 GMT 2010


Hi Reuti,

$fs_stdin_file_staging is one of the prolog arguments.  As part of the prolog script, I write out $fs_stdin_file_staging and the other $fs_ variables.  They are all '0' or "", even when I use the python script below to submit the job using drmaa.

Thanks,

Sum Yee
---------------------------------------

qconf -sq default.q
qname                 default.q
hostlist              @allhosts
seq_no                10
load_thresholds       NONE
suspend_thresholds    NONE
nsuspend              1
suspend_interval      00:05:00
priority              0
min_cpu_interval      00:05:00
processors            UNDEFINED
qtype                 BATCH INTERACTIVE
ckpt_list             reloc
pe_list               make
rerun                 FALSE
slots                 4
tmpdir                /tmp/sge_tmp
shell                 /bin/bash
prolog                /usr/local/GridEngine/custom/bin/prolog.py \
                      $fs_stdin_file_staging $fs_stdin_host $fs_stdin_path
epilog                /usr/local/GridEngine/custom/bin/epilog.py \
                      $fs_stdout_file_staging $fs_stdout_host $fs_stdout_path
shell_start_mode      unix_behavior
starter_method        /usr/local/GridEngine/custom/bin/starter.py
suspend_method        NONE
resume_method         NONE
terminate_method      NONE
notify                00:05:00
owner_list            NONE
user_lists            NONE
xuser_lists           NONE
subordinate_list      slots=4(low.q:0:lr)

-----Original Message-----
From: reuti [mailto:reuti at staff.uni-marburg.de] 
Sent: Wednesday, March 17, 2010 10:12 AM
To: users at gridengine.sunsource.net
Subject: Re: [GE users] Delegated filestaging problem

Hi,

Am 17.03.2010 um 17:54 schrieb sumyee:

> Hi,
>
> I am trying to use delegated filestaging to transfer files.  Even  
> though I have delegated filestaging turned on and I am submitting  
> jobs via drmaa, $fs_stdin_file_staging is always '0'.  Is there  
> something else I need to do to run delegated filestaging?

when is $fs_stdin_file_staging always 0? It's not clear where you  
check it. It's a pseudo variable which can be in the definition of  
the prolog/epilog. It's not to be used in the job script.

-- Reuti


>
> Thanks!
>
> Sum
>
>
> Script to submit job:
> #!/usr/bin/env python
>
> import os, sys, drmaa
>
> def main():
> 	s = drmaa.Session()
> 	s.initialize()
>
> 	jt = s.createJobTemplate()
> 	jt.remoteCommand = os.getcwd() + '/simple.sh'
> 	path = '%s:%s' % (os.uname()[1], os.getcwd())
> 	jt.drmaa_input_path=path
> 	jt.drmaa_transfer_files='io'
> 	jobid = s.runJob(jt)
> 	s.deleteJobTemplate(jt)
> 	s.exit()
>
> if __name__ == '__main__':
> 	main()
>
> ---------------------------------------------------------------------- 
> --------------------
> qconf -sconf
> #global:
> execd_spool_dir              /usr/local/GridEngine/default/spool
> mailer                       /bin/mail
> xterm                        /usr/bin/X11/xterm
> load_sensor                  /usr/local/GridEngine/custom/bin/load.py
> prolog                       none
> epilog                       none
> shell_start_mode             unix_behavior
> login_shells                 sh,ksh,csh,tcsh,bash
> min_uid                      0
> min_gid                      0
> user_lists                   none
> xuser_lists                  none
> projects                     none
> xprojects                    none
> enforce_project              false
> enforce_user                 auto
> load_report_time             00:00:40
> max_unheard                  00:05:00
> reschedule_unknown           00:00:00
> loglevel                     log_warning
> administrator_mail           none
> set_token_cmd                none
> pag_cmd                      none
> token_extend_time            none
> shepherd_cmd                 none
> qmaster_params               MAX_DYN_EC=10000,ENABLE_FORCED_QDEL=true
> reporting_params             accounting=true reporting=true \
>                              flush_time=00:00:15 joblog=false  
> sharelog=00:00:00
> finished_jobs                100
> gid_range                    20000-30000
> qlogin_command               builtin
> qlogin_daemon                builtin
> rlogin_command               /usr/bin/ssh -t -Y
> rlogin_daemon                /usr/sbin/sshd -i
> rsh_command                  /usr/bin/ssh -t -Y
> rsh_daemon                   /usr/sbin/sshd -i
> max_aj_instances             2000
> max_aj_tasks                 75000
> max_u_jobs                   0
> max_jobs                     0
> max_advance_reservations     15
> auto_user_oticket            0
> auto_user_fshare             0
> auto_user_default_project    none
> auto_user_delete_time        86400
> delegated_file_staging       true
> reprioritize                 false
> jsv_url                      none
> jsv_allowed_mod              ac,h,i,e,o,j,M,N,p,w,l
> execd_params                 INHERIT_ENV=false SET_LIB_PATH=true
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do? 
> dsForumId=38&dsMessageId=249193
>
> To unsubscribe from this discussion, e-mail: [users- 
> unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=249196

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=249210

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list