[GE users] Environment variables

Reuti reuti at staff.uni-marburg.de
Wed Jan 19 11:48:41 GMT 2005


Where did you put the wrapper on the slave node (oder did you modify the 
user entries to use your wrapper)?

Another idea: test in the /etc/profile.local (for SuSE) for the node and 
the user and set an ulimit, to limit the CPU time. This way users can 
login and look around, but can't do any interactive work. - Reuti

Wheeler, Dr M.D. wrote:
> OK, so here goes the latest, I can using the script below initiate a serial job on a particular node using SGE while not alllowing interactive user access on the node.
> 
> 
> #!/bin/bash
> #
> #wrapper script to provide shell access to frontend
> #while denying access to compute nodes
> #
> 
> FRONTEND_HOSTNAME="my.fqdn"
> 
> if [ "`hostname`" = "$FRONTEND_HOSTNAME" ]; then
>     #provide a bash login shell
>     /bin/bash -l
> else
>     #politely refuse login and exit
>     echo ""
>     echo "############################################################################"
>     echo "You do not have permission to login on compute node `hostname`"
>     echo "############################################################################"
>     echo ""
>     sleep 1
>     exit 0
> fi
> 
> exit 0
> 
> 
> However, when I start a parallel job is when the problems start, my guess is that when parallel threads are initiated they are started with using my shell wrapper rather than /bin/bash.  My pe start proc is:
> 
> #!/bin/sh
> #
> #
> # preparation of the mpi machine file
> #
> # usage: startmpi.sh [options] <pe_hostfile>
> #
> #        options are:
> #                     -catch_hostname
> #                      force use of hostname wrapper in $TMPDIR when starting mpirun
> #                     -catch_rsh
> #                      force use of rsh wrapper in $TMPDIR when starting mpirun
> #                     -unique
> #                      generate a machinefile where each hostname appears only once
> #                      This is needed to setup a multithreaded mpi application
> #
> 
> PeHostfile2MachineFile()
> {
>    cat $1 | while read line; do
>       # echo $line
>       host=`echo $line|cut -f1 -d" "|cut -f1 -d"."`
>       nslots=`echo $line|cut -f2 -d" "`
>       i=1
>       while [ $i -le $nslots ]; do
>          # add here code to map regular hostnames into ATM hostnames
>          echo $host.local
>          i=`expr $i + 1`
>       done
>    done
> }
> 
> 
> #
> # startup of MPI conforming with the Grid Engine
> # Parallel Environment interface
> #
> # on success the job will find a machine-file in $TMPDIR/machines
> # useful to control parameters passed to us
> echo $*
> 
> # parse options
> catch_rsh=0
> catch_hostname=0
> unique=0
> while [ "$1" != "" ]; do
>    case "$1" in
>       -catch_rsh)
>          catch_rsh=1
>          ;;
>       -catch_hostname)
>          catch_hostname=1
>          ;;
>       -unique)
>          unique=1
>          ;;
>       *)
>          break;
>          ;;
>    esac
>    shift
> done
> 
> me=`basename $0`
> 
> # test number of args
> if [ $# -ne 1 ]; then
>    echo "$me: got wrong number of arguments" >&2
>    exit 1
> fi
> 
> # get arguments
> pe_hostfile=$1
> 
> # ensure pe_hostfile is readable
> if [ ! -r $pe_hostfile ]; then
>    echo "$me: can't read $pe_hostfile" >&2
>    exit 1
> fi
> # create machine-file
> # remove column with number of slots per queue
> # mpi does not support them in this form
> machines="$TMPDIR/machines"
> 
> if [ $unique = 1 ]; then
>    PeHostfile2MachineFile $pe_hostfile | uniq >> $machines
> else
>    PeHostfile2MachineFile $pe_hostfile >> $machines
> fi
> 
> # trace machines file
> cat $machines
> 
> # Making job subdirectories on all nodes
> for node in `cat $TMPDIR/machines` ; do
>   $SGE_ROOT/bin/$ARC/qrsh -inherit $node "[ -d \"/state/partition1/$JOB_ID\" ] || mkdir -m 700 /stat
> e/partition{1,2}/$JOB_ID"
> done
> 
> #
> # Make script wrapper for 'rsh' available in jobs tmp dir
> #
> if [ $catch_rsh = 1 ]; then
>    rsh_wrapper=$SGE_ROOT/mpi/rsh
>    if [ ! -x $rsh_wrapper ]; then
>       echo "$me: can't execute $rsh_wrapper" >&2
>       echo "     maybe it resides at a file system not available at this machine" >&2
>       exit 1
>    fi
> 
>    rshcmd=rsh
>    case "$ARC" in
>       hp|hp10|hp11|hp11-64) rshcmd=remsh ;;
>       *) ;;
>    esac
>    # note: This could also be done using rcp, ftp or s.th.
>    #       else. We use a symbolic link since it is the
>    #       cheapest in case of a shared filesystem
>    #
>    ln -s $rsh_wrapper $TMPDIR/$rshcmd
> fi
> 
> #
> # Make script wrapper for 'hostname' available in jobs tmp dir
> #
> if [ $catch_hostname = 1 ]; then
>    hostname_wrapper=$SGE_ROOT/mpi/hostname
>    if [ ! -x $hostname_wrapper ]; then
>       echo "$me: can't execute $hostname_wrapper" >&2
>       echo "     maybe it resides at a file system not available at this machine" >&2
>       exit 1
>    fi
> 
>    # note: This could also be done using rcp, ftp or s.th.
>    #       else. We use a symbolic link since it is the
>    #       cheapest in case of a shared filesystem
>    #
>    ln -s $hostname_wrapper $TMPDIR/hostname
> fi
> 
> # signal success to caller
> exit 0
> 
> 
> 
> ----------------------------------------------
> Dr. Martyn D. Wheeler
> Department of Chemistry
> University of Leicester
> University Road
> Leicester, LE1 7RH, UK.
> Tel (office): +44 (0)116 252 3985
> Tel (lab):    +44 (0)116 252 2115
> Fax:          +44 (0)116 252 3789
> Email:        martyn.wheeler at le.ac.uk
> http://www.le.ac.uk/chemistry/staff/mdw10.html
>  
> 
> 
>>-----Original Message-----
>>From: Reuti [mailto:reuti at staff.uni-marburg.de]
>>Sent: 19 January 2005 09:43
>>To: users at gridengine.sunsource.net
>>Subject: RE: [GE users] Environment variables
>>
>>
>>Hi,
>>
>>just to step into this discussion: is ROCKS copying the head 
>>node file sytem to 
>>the slave nodes or always starting in some kind of NFS mode? 
>>It is also 
>>possible to change the file for the tcpip services and ignore 
>>rsh/ssh requests 
>>completely (in SuSE it's /etc/inetd.conf or with newer 
>>installations inside the 
>>xinetd.d/rsh file). Only on the slave nodes of course, maybe 
>>with a "sed" 
>>command during the first startup to alter this setup file (if 
>>it's just a copy 
>>from the head node).
>>
>>Cheers - Reuti
>>
>>
>>Quoting "Wheeler, Dr M.D." <mdw10 at leicester.ac.uk>:
>>
>>
>>>Thanks, maybe you could help me further, I am trying to 
>>
>>setup a wrapper
>>
>>>script that determines whether a user can login to a 
>>
>>machine or not (I would
>>
>>>like users to be able to have inrteractive access to the 
>>
>>frontend machine but
>>
>>>not the compute nodes).  So far I have this
>>>
>>>#!/bin/bash
>>>#
>>>#wrapper script to provide shell access to frontend
>>>#while denying access to compute nodes
>>>#
>>>
>>>FRONTEND_HOSTNAME="my.fqdn.net"
>>>
>>>if [ "`hostname`" = "$FRONTEND_HOSTNAME" ] || [ -n "$JOB_ID" ]; then
>>>    #provide a bash login shell
>>>    /bin/bash -l
>>>
>>>else
>>>    #politely refuse login and exit
>>>    echo ""
>>>    echo
>>>
>>
>>"#############################################################
>>###############"
>>
>>>    echo "You do not have permission to login on compute 
>>
>>node `hostname`"
>>
>>>    echo
>>>
>>
>>"#############################################################
>>###############"
>>
>>>    echo ""
>>>    sleep 1
>>>    exit 0
>>>fi
>>>
>>>exit 0
>>>
>>>however, while this prevents users from loggin in to a 
>>
>>machine, it also seems
>>
>>>to be preventing them submitting jobs too.
>>>Any ideas how I might make this work?
>>>
>>>Martyn
>>>
>>>----------------------------------------------
>>>Dr. Martyn D. Wheeler
>>>Department of Chemistry
>>>University of Leicester
>>>University Road
>>>Leicester, LE1 7RH, UK.
>>>Tel (office): +44 (0)116 252 3985
>>>Tel (lab):    +44 (0)116 252 2115
>>>Fax:          +44 (0)116 252 3789
>>>Email:        martyn.wheeler at le.ac.uk
>>>http://www.le.ac.uk/chemistry/staff/mdw10.html
>>> 
>>>
>>>
>>>>-----Original Message-----
>>>>From: Ron Chen [mailto:ron_chen_123 at yahoo.com]
>>>>Sent: 19 January 2005 08:35
>>>>To: users at gridengine.sunsource.net
>>>>Subject: Re: [GE users] Environment variables
>>>>
>>>>
>>>>I like this one: $JOB_ID
>>>>
>>>>Actually, there are many variables setup by SGE, you
>>>>can submit a job to print all the env. vars.:
>>>>
>>>>==========
>>>>#!/bin/sh
>>>>
>>>>env
>>>>==========
>>>>
>>>> -Ron
>>>>
>>>>--- "Wheeler, Dr M.D." <mdw10 at leicester.ac.uk> wrote:
>>>>
>>>>>Is there a way of distinguishing when a user is
>>>>>running a job on a node via SGE and when a user is
>>>>>simply trying to log in to a node.  Maybe there is
>>>>>an environment variable set up or soemthing?  I'd
>>>>>like to find a simple way of distinguishing between
>>>>>interactive logins and jobs submitted via SGE
>>>>>
>>>>>Thanks
>>>>>MArtyn
>>>>>
>>>>>----------------------------------------------
>>>>>Dr. Martyn D. Wheeler
>>>>>Department of Chemistry
>>>>>University of Leicester
>>>>>University Road
>>>>>Leicester, LE1 7RH, UK.
>>>>>Tel (office): +44 (0)116 252 3985
>>>>>Tel (lab):    +44 (0)116 252 2115
>>>>>Fax:          +44 (0)116 252 3789
>>>>>Email:        martyn.wheeler at le.ac.uk
>>>>>http://www.le.ac.uk/chemistry/staff/mdw10.html
>>>>> 
>>>>>
>>>>>
>>>>
>>---------------------------------------------------------------------
>>
>>>>>To unsubscribe, e-mail:
>>>>>users-unsubscribe at gridengine.sunsource.net
>>>>>For additional commands, e-mail:
>>>>>users-help at gridengine.sunsource.net
>>>>>
>>>>>
>>>>
>>>>
>>>>
>>>>		
>>>>__________________________________ 
>>>>Do you Yahoo!? 
>>>>Yahoo! Mail - 250MB free storage. Do more. Manage less. 
>>>>http://info.mail.yahoo.com/mail_250
>>>>
>>>>
>>
>>---------------------------------------------------------------------
>>
>>>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>>>For additional commands, e-mail: 
>>
>>users-help at gridengine.sunsource.net
>>
>>>>
>>>
>>---------------------------------------------------------------------
>>
>>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>
>>>
>>
>>
>>
>>---------------------------------------------------------------------
>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list