[GE users] Environment variables

Wheeler, Dr M.D. mdw10 at leicester.ac.uk
Wed Jan 19 12:44:16 GMT 2005


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

The problem with ulimit is that when a parallel thread is run it inherits ulimit from the shell and runs out of time.

At present the only solution I have found is to set a profile script in /etc/profile.d/

#This script sets up environment variables on the nodes
#
FRONTEND_HOSTNAME="**.**.**.**.**"

if [ "`hostname`" = "$FRONTEND_HOSTNAME" ] || [ "$UID" = 0 ]; then

#set interactive shell aliases
    alias la='ls -ha'
    alias ll='ls -lha'
    alias ls='ls --color=yes'
    alias xterm='xterm &'
    alias psg='ps -aux | grep \!* | grep -v grep'
    alias mroe='more'

#set interactive CPU time
    if [ "$UID" != "0" ]; then
        ulimit -t 300 # set interactive cputime
    fi

#set environment paths for PGI compiler
    export PGI=/home/software/pgi
    export PATH=$PGI/linux86-64/5.2/bin:$PATH
    export MANPATH=$MANPATH:$PGI/linux86-64/5.2/man

#set path to find other software
    export PATH=$PATH:/home/software/bin
    export EDITOR=pico
#    export LD_LIBRARY_PATH=/usr/local/pgplot
#    export PGPLOT_DIR=/usr/local/pgplot

else

    export TMOUT=1

fi

however, it is fairly trvial to re-export TMOUT=0 if you paste it into the window quickly

Martyn

----------------------------------------------
Dr. Martyn D. Wheeler
Department of Chemistry
University of Leicester
University Road
Leicester, LE1 7RH, UK.
Tel (office): +44 (0)116 252 3985
Tel (lab):    +44 (0)116 252 2115
Fax:          +44 (0)116 252 3789
Email:        martyn.wheeler at le.ac.uk
http://www.le.ac.uk/chemistry/staff/mdw10.html
 

> -----Original Message-----
> From: Reuti [mailto:reuti at staff.uni-marburg.de]
> Sent: 19 January 2005 11:49
> To: users at gridengine.sunsource.net
> Subject: Re: [GE users] Environment variables
> 
> 
> Where did you put the wrapper on the slave node (oder did you 
> modify the 
> user entries to use your wrapper)?
> 
> Another idea: test in the /etc/profile.local (for SuSE) for 
> the node and 
> the user and set an ulimit, to limit the CPU time. This way users can 
> login and look around, but can't do any interactive work. - Reuti
> 
> Wheeler, Dr M.D. wrote:
> > OK, so here goes the latest, I can using the script below 
> initiate a serial job on a particular node using SGE while 
> not alllowing interactive user access on the node.
> > 
> > 
> > #!/bin/bash
> > #
> > #wrapper script to provide shell access to frontend
> > #while denying access to compute nodes
> > #
> > 
> > FRONTEND_HOSTNAME="my.fqdn"
> > 
> > if [ "`hostname`" = "$FRONTEND_HOSTNAME" ]; then
> >     #provide a bash login shell
> >     /bin/bash -l
> > else
> >     #politely refuse login and exit
> >     echo ""
> >     echo 
> "#############################################################
> ###############"
> >     echo "You do not have permission to login on compute 
> node `hostname`"
> >     echo 
> "#############################################################
> ###############"
> >     echo ""
> >     sleep 1
> >     exit 0
> > fi
> > 
> > exit 0
> > 
> > 
> > However, when I start a parallel job is when the problems 
> start, my guess is that when parallel threads are initiated 
> they are started with using my shell wrapper rather than 
> /bin/bash.  My pe start proc is:
> > 
> > #!/bin/sh
> > #
> > #
> > # preparation of the mpi machine file
> > #
> > # usage: startmpi.sh [options] <pe_hostfile>
> > #
> > #        options are:
> > #                     -catch_hostname
> > #                      force use of hostname wrapper in 
> $TMPDIR when starting mpirun
> > #                     -catch_rsh
> > #                      force use of rsh wrapper in $TMPDIR 
> when starting mpirun
> > #                     -unique
> > #                      generate a machinefile where each 
> hostname appears only once
> > #                      This is needed to setup a 
> multithreaded mpi application
> > #
> > 
> > PeHostfile2MachineFile()
> > {
> >    cat $1 | while read line; do
> >       # echo $line
> >       host=`echo $line|cut -f1 -d" "|cut -f1 -d"."`
> >       nslots=`echo $line|cut -f2 -d" "`
> >       i=1
> >       while [ $i -le $nslots ]; do
> >          # add here code to map regular hostnames into ATM hostnames
> >          echo $host.local
> >          i=`expr $i + 1`
> >       done
> >    done
> > }
> > 
> > 
> > #
> > # startup of MPI conforming with the Grid Engine
> > # Parallel Environment interface
> > #
> > # on success the job will find a machine-file in $TMPDIR/machines
> > # useful to control parameters passed to us
> > echo $*
> > 
> > # parse options
> > catch_rsh=0
> > catch_hostname=0
> > unique=0
> > while [ "$1" != "" ]; do
> >    case "$1" in
> >       -catch_rsh)
> >          catch_rsh=1
> >          ;;
> >       -catch_hostname)
> >          catch_hostname=1
> >          ;;
> >       -unique)
> >          unique=1
> >          ;;
> >       *)
> >          break;
> >          ;;
> >    esac
> >    shift
> > done
> > 
> > me=`basename $0`
> > 
> > # test number of args
> > if [ $# -ne 1 ]; then
> >    echo "$me: got wrong number of arguments" >&2
> >    exit 1
> > fi
> > 
> > # get arguments
> > pe_hostfile=$1
> > 
> > # ensure pe_hostfile is readable
> > if [ ! -r $pe_hostfile ]; then
> >    echo "$me: can't read $pe_hostfile" >&2
> >    exit 1
> > fi
> > # create machine-file
> > # remove column with number of slots per queue
> > # mpi does not support them in this form
> > machines="$TMPDIR/machines"
> > 
> > if [ $unique = 1 ]; then
> >    PeHostfile2MachineFile $pe_hostfile | uniq >> $machines
> > else
> >    PeHostfile2MachineFile $pe_hostfile >> $machines
> > fi
> > 
> > # trace machines file
> > cat $machines
> > 
> > # Making job subdirectories on all nodes
> > for node in `cat $TMPDIR/machines` ; do
> >   $SGE_ROOT/bin/$ARC/qrsh -inherit $node "[ -d 
> \"/state/partition1/$JOB_ID\" ] || mkdir -m 700 /stat
> > e/partition{1,2}/$JOB_ID"
> > done
> > 
> > #
> > # Make script wrapper for 'rsh' available in jobs tmp dir
> > #
> > if [ $catch_rsh = 1 ]; then
> >    rsh_wrapper=$SGE_ROOT/mpi/rsh
> >    if [ ! -x $rsh_wrapper ]; then
> >       echo "$me: can't execute $rsh_wrapper" >&2
> >       echo "     maybe it resides at a file system not 
> available at this machine" >&2
> >       exit 1
> >    fi
> > 
> >    rshcmd=rsh
> >    case "$ARC" in
> >       hp|hp10|hp11|hp11-64) rshcmd=remsh ;;
> >       *) ;;
> >    esac
> >    # note: This could also be done using rcp, ftp or s.th.
> >    #       else. We use a symbolic link since it is the
> >    #       cheapest in case of a shared filesystem
> >    #
> >    ln -s $rsh_wrapper $TMPDIR/$rshcmd
> > fi
> > 
> > #
> > # Make script wrapper for 'hostname' available in jobs tmp dir
> > #
> > if [ $catch_hostname = 1 ]; then
> >    hostname_wrapper=$SGE_ROOT/mpi/hostname
> >    if [ ! -x $hostname_wrapper ]; then
> >       echo "$me: can't execute $hostname_wrapper" >&2
> >       echo "     maybe it resides at a file system not 
> available at this machine" >&2
> >       exit 1
> >    fi
> > 
> >    # note: This could also be done using rcp, ftp or s.th.
> >    #       else. We use a symbolic link since it is the
> >    #       cheapest in case of a shared filesystem
> >    #
> >    ln -s $hostname_wrapper $TMPDIR/hostname
> > fi
> > 
> > # signal success to caller
> > exit 0
> > 
> > 
> > 
> > ----------------------------------------------
> > Dr. Martyn D. Wheeler
> > Department of Chemistry
> > University of Leicester
> > University Road
> > Leicester, LE1 7RH, UK.
> > Tel (office): +44 (0)116 252 3985
> > Tel (lab):    +44 (0)116 252 2115
> > Fax:          +44 (0)116 252 3789
> > Email:        martyn.wheeler at le.ac.uk
> > http://www.le.ac.uk/chemistry/staff/mdw10.html
> >  
> > 
> > 
> >>-----Original Message-----
> >>From: Reuti [mailto:reuti at staff.uni-marburg.de]
> >>Sent: 19 January 2005 09:43
> >>To: users at gridengine.sunsource.net
> >>Subject: RE: [GE users] Environment variables
> >>
> >>
> >>Hi,
> >>
> >>just to step into this discussion: is ROCKS copying the head 
> >>node file sytem to 
> >>the slave nodes or always starting in some kind of NFS mode? 
> >>It is also 
> >>possible to change the file for the tcpip services and ignore 
> >>rsh/ssh requests 
> >>completely (in SuSE it's /etc/inetd.conf or with newer 
> >>installations inside the 
> >>xinetd.d/rsh file). Only on the slave nodes of course, maybe 
> >>with a "sed" 
> >>command during the first startup to alter this setup file (if 
> >>it's just a copy 
> >>from the head node).
> >>
> >>Cheers - Reuti
> >>
> >>
> >>Quoting "Wheeler, Dr M.D." <mdw10 at leicester.ac.uk>:
> >>
> >>
> >>>Thanks, maybe you could help me further, I am trying to 
> >>
> >>setup a wrapper
> >>
> >>>script that determines whether a user can login to a 
> >>
> >>machine or not (I would
> >>
> >>>like users to be able to have inrteractive access to the 
> >>
> >>frontend machine but
> >>
> >>>not the compute nodes).  So far I have this
> >>>
> >>>#!/bin/bash
> >>>#
> >>>#wrapper script to provide shell access to frontend
> >>>#while denying access to compute nodes
> >>>#
> >>>
> >>>FRONTEND_HOSTNAME="my.fqdn.net"
> >>>
> >>>if [ "`hostname`" = "$FRONTEND_HOSTNAME" ] || [ -n 
> "$JOB_ID" ]; then
> >>>    #provide a bash login shell
> >>>    /bin/bash -l
> >>>
> >>>else
> >>>    #politely refuse login and exit
> >>>    echo ""
> >>>    echo
> >>>
> >>
> >>"#############################################################
> >>###############"
> >>
> >>>    echo "You do not have permission to login on compute 
> >>
> >>node `hostname`"
> >>
> >>>    echo
> >>>
> >>
> >>"#############################################################
> >>###############"
> >>
> >>>    echo ""
> >>>    sleep 1
> >>>    exit 0
> >>>fi
> >>>
> >>>exit 0
> >>>
> >>>however, while this prevents users from loggin in to a 
> >>
> >>machine, it also seems
> >>
> >>>to be preventing them submitting jobs too.
> >>>Any ideas how I might make this work?
> >>>
> >>>Martyn
> >>>
> >>>----------------------------------------------
> >>>Dr. Martyn D. Wheeler
> >>>Department of Chemistry
> >>>University of Leicester
> >>>University Road
> >>>Leicester, LE1 7RH, UK.
> >>>Tel (office): +44 (0)116 252 3985
> >>>Tel (lab):    +44 (0)116 252 2115
> >>>Fax:          +44 (0)116 252 3789
> >>>Email:        martyn.wheeler at le.ac.uk
> >>>http://www.le.ac.uk/chemistry/staff/mdw10.html
> >>> 
> >>>
> >>>
> >>>>-----Original Message-----
> >>>>From: Ron Chen [mailto:ron_chen_123 at yahoo.com]
> >>>>Sent: 19 January 2005 08:35
> >>>>To: users at gridengine.sunsource.net
> >>>>Subject: Re: [GE users] Environment variables
> >>>>
> >>>>
> >>>>I like this one: $JOB_ID
> >>>>
> >>>>Actually, there are many variables setup by SGE, you
> >>>>can submit a job to print all the env. vars.:
> >>>>
> >>>>==========
> >>>>#!/bin/sh
> >>>>
> >>>>env
> >>>>==========
> >>>>
> >>>> -Ron
> >>>>
> >>>>--- "Wheeler, Dr M.D." <mdw10 at leicester.ac.uk> wrote:
> >>>>
> >>>>>Is there a way of distinguishing when a user is
> >>>>>running a job on a node via SGE and when a user is
> >>>>>simply trying to log in to a node.  Maybe there is
> >>>>>an environment variable set up or soemthing?  I'd
> >>>>>like to find a simple way of distinguishing between
> >>>>>interactive logins and jobs submitted via SGE
> >>>>>
> >>>>>Thanks
> >>>>>MArtyn
> >>>>>
> >>>>>----------------------------------------------
> >>>>>Dr. Martyn D. Wheeler
> >>>>>Department of Chemistry
> >>>>>University of Leicester
> >>>>>University Road
> >>>>>Leicester, LE1 7RH, UK.
> >>>>>Tel (office): +44 (0)116 252 3985
> >>>>>Tel (lab):    +44 (0)116 252 2115
> >>>>>Fax:          +44 (0)116 252 3789
> >>>>>Email:        martyn.wheeler at le.ac.uk
> >>>>>http://www.le.ac.uk/chemistry/staff/mdw10.html
> >>>>> 
> >>>>>
> >>>>>
> >>>>
> >>------------------------------------------------------------
> ---------
> >>
> >>>>>To unsubscribe, e-mail:
> >>>>>users-unsubscribe at gridengine.sunsource.net
> >>>>>For additional commands, e-mail:
> >>>>>users-help at gridengine.sunsource.net
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>>		
> >>>>__________________________________ 
> >>>>Do you Yahoo!? 
> >>>>Yahoo! Mail - 250MB free storage. Do more. Manage less. 
> >>>>http://info.mail.yahoo.com/mail_250
> >>>>
> >>>>
> >>
> >>------------------------------------------------------------
> ---------
> >>
> >>>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>>>For additional commands, e-mail: 
> >>
> >>users-help at gridengine.sunsource.net
> >>
> >>>>
> >>>
> >>------------------------------------------------------------
> ---------
> >>
> >>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>>For additional commands, e-mail: 
> users-help at gridengine.sunsource.net
> >>>
> >>>
> >>
> >>
> >>
> >>------------------------------------------------------------
> ---------
> >>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> >>For additional commands, e-mail: users-help at gridengine.sunsource.net
> >>
> >>
> > 
> > 
> > 
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> > 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list