[GE users] Environment variables

Wheeler, Dr M.D. mdw10 at leicester.ac.uk
Wed Jan 19 10:50:31 GMT 2005


    [ The following text is in the "iso-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

OK, so here goes the latest, I can using the script below initiate a serial job on a particular node using SGE while not alllowing interactive user access on the node.


#!/bin/bash
#
#wrapper script to provide shell access to frontend
#while denying access to compute nodes
#

FRONTEND_HOSTNAME="my.fqdn"

if [ "`hostname`" = "$FRONTEND_HOSTNAME" ]; then
    #provide a bash login shell
    /bin/bash -l
else
    #politely refuse login and exit
    echo ""
    echo "############################################################################"
    echo "You do not have permission to login on compute node `hostname`"
    echo "############################################################################"
    echo ""
    sleep 1
    exit 0
fi

exit 0


However, when I start a parallel job is when the problems start, my guess is that when parallel threads are initiated they are started with using my shell wrapper rather than /bin/bash.  My pe start proc is:

#!/bin/sh
#
#
# preparation of the mpi machine file
#
# usage: startmpi.sh [options] <pe_hostfile>
#
#        options are:
#                     -catch_hostname
#                      force use of hostname wrapper in $TMPDIR when starting mpirun
#                     -catch_rsh
#                      force use of rsh wrapper in $TMPDIR when starting mpirun
#                     -unique
#                      generate a machinefile where each hostname appears only once
#                      This is needed to setup a multithreaded mpi application
#

PeHostfile2MachineFile()
{
   cat $1 | while read line; do
      # echo $line
      host=`echo $line|cut -f1 -d" "|cut -f1 -d"."`
      nslots=`echo $line|cut -f2 -d" "`
      i=1
      while [ $i -le $nslots ]; do
         # add here code to map regular hostnames into ATM hostnames
         echo $host.local
         i=`expr $i + 1`
      done
   done
}


#
# startup of MPI conforming with the Grid Engine
# Parallel Environment interface
#
# on success the job will find a machine-file in $TMPDIR/machines
# useful to control parameters passed to us
echo $*

# parse options
catch_rsh=0
catch_hostname=0
unique=0
while [ "$1" != "" ]; do
   case "$1" in
      -catch_rsh)
         catch_rsh=1
         ;;
      -catch_hostname)
         catch_hostname=1
         ;;
      -unique)
         unique=1
         ;;
      *)
         break;
         ;;
   esac
   shift
done

me=`basename $0`

# test number of args
if [ $# -ne 1 ]; then
   echo "$me: got wrong number of arguments" >&2
   exit 1
fi

# get arguments
pe_hostfile=$1

# ensure pe_hostfile is readable
if [ ! -r $pe_hostfile ]; then
   echo "$me: can't read $pe_hostfile" >&2
   exit 1
fi
# create machine-file
# remove column with number of slots per queue
# mpi does not support them in this form
machines="$TMPDIR/machines"

if [ $unique = 1 ]; then
   PeHostfile2MachineFile $pe_hostfile | uniq >> $machines
else
   PeHostfile2MachineFile $pe_hostfile >> $machines
fi

# trace machines file
cat $machines

# Making job subdirectories on all nodes
for node in `cat $TMPDIR/machines` ; do
  $SGE_ROOT/bin/$ARC/qrsh -inherit $node "[ -d \"/state/partition1/$JOB_ID\" ] || mkdir -m 700 /stat
e/partition{1,2}/$JOB_ID"
done

#
# Make script wrapper for 'rsh' available in jobs tmp dir
#
if [ $catch_rsh = 1 ]; then
   rsh_wrapper=$SGE_ROOT/mpi/rsh
   if [ ! -x $rsh_wrapper ]; then
      echo "$me: can't execute $rsh_wrapper" >&2
      echo "     maybe it resides at a file system not available at this machine" >&2
      exit 1
   fi

   rshcmd=rsh
   case "$ARC" in
      hp|hp10|hp11|hp11-64) rshcmd=remsh ;;
      *) ;;
   esac
   # note: This could also be done using rcp, ftp or s.th.
   #       else. We use a symbolic link since it is the
   #       cheapest in case of a shared filesystem
   #
   ln -s $rsh_wrapper $TMPDIR/$rshcmd
fi

#
# Make script wrapper for 'hostname' available in jobs tmp dir
#
if [ $catch_hostname = 1 ]; then
   hostname_wrapper=$SGE_ROOT/mpi/hostname
   if [ ! -x $hostname_wrapper ]; then
      echo "$me: can't execute $hostname_wrapper" >&2
      echo "     maybe it resides at a file system not available at this machine" >&2
      exit 1
   fi

   # note: This could also be done using rcp, ftp or s.th.
   #       else. We use a symbolic link since it is the
   #       cheapest in case of a shared filesystem
   #
   ln -s $hostname_wrapper $TMPDIR/hostname
fi

# signal success to caller
exit 0



----------------------------------------------
Dr. Martyn D. Wheeler
Department of Chemistry
University of Leicester
University Road
Leicester, LE1 7RH, UK.
Tel (office): +44 (0)116 252 3985
Tel (lab):    +44 (0)116 252 2115
Fax:          +44 (0)116 252 3789
Email:        martyn.wheeler at le.ac.uk
http://www.le.ac.uk/chemistry/staff/mdw10.html
 

> -----Original Message-----
> From: Reuti [mailto:reuti at staff.uni-marburg.de]
> Sent: 19 January 2005 09:43
> To: users at gridengine.sunsource.net
> Subject: RE: [GE users] Environment variables
> 
> 
> Hi,
> 
> just to step into this discussion: is ROCKS copying the head 
> node file sytem to 
> the slave nodes or always starting in some kind of NFS mode? 
> It is also 
> possible to change the file for the tcpip services and ignore 
> rsh/ssh requests 
> completely (in SuSE it's /etc/inetd.conf or with newer 
> installations inside the 
> xinetd.d/rsh file). Only on the slave nodes of course, maybe 
> with a "sed" 
> command during the first startup to alter this setup file (if 
> it's just a copy 
> from the head node).
> 
> Cheers - Reuti
> 
> 
> Quoting "Wheeler, Dr M.D." <mdw10 at leicester.ac.uk>:
> 
> > Thanks, maybe you could help me further, I am trying to 
> setup a wrapper
> > script that determines whether a user can login to a 
> machine or not (I would
> > like users to be able to have inrteractive access to the 
> frontend machine but
> > not the compute nodes).  So far I have this
> > 
> > #!/bin/bash
> > #
> > #wrapper script to provide shell access to frontend
> > #while denying access to compute nodes
> > #
> > 
> > FRONTEND_HOSTNAME="my.fqdn.net"
> > 
> > if [ "`hostname`" = "$FRONTEND_HOSTNAME" ] || [ -n "$JOB_ID" ]; then
> >     #provide a bash login shell
> >     /bin/bash -l
> > 
> > else
> >     #politely refuse login and exit
> >     echo ""
> >     echo
> > 
> "#############################################################
> ###############"
> >     echo "You do not have permission to login on compute 
> node `hostname`"
> >     echo
> > 
> "#############################################################
> ###############"
> >     echo ""
> >     sleep 1
> >     exit 0
> > fi
> > 
> > exit 0
> > 
> > however, while this prevents users from loggin in to a 
> machine, it also seems
> > to be preventing them submitting jobs too.
> > Any ideas how I might make this work?
> > 
> > Martyn
> > 
> > ----------------------------------------------
> > Dr. Martyn D. Wheeler
> > Department of Chemistry
> > University of Leicester
> > University Road
> > Leicester, LE1 7RH, UK.
> > Tel (office): +44 (0)116 252 3985
> > Tel (lab):    +44 (0)116 252 2115
> > Fax:          +44 (0)116 252 3789
> > Email:        martyn.wheeler at le.ac.uk
> > http://www.le.ac.uk/chemistry/staff/mdw10.html
> >  
> > 
> > > -----Original Message-----
> > > From: Ron Chen [mailto:ron_chen_123 at yahoo.com]
> > > Sent: 19 January 2005 08:35
> > > To: users at gridengine.sunsource.net
> > > Subject: Re: [GE users] Environment variables
> > > 
> > > 
> > > I like this one: $JOB_ID
> > > 
> > > Actually, there are many variables setup by SGE, you
> > > can submit a job to print all the env. vars.:
> > > 
> > > ==========
> > > #!/bin/sh
> > > 
> > > env
> > > ==========
> > > 
> > >  -Ron
> > > 
> > > --- "Wheeler, Dr M.D." <mdw10 at leicester.ac.uk> wrote:
> > > > Is there a way of distinguishing when a user is
> > > > running a job on a node via SGE and when a user is
> > > > simply trying to log in to a node.  Maybe there is
> > > > an environment variable set up or soemthing?  I'd
> > > > like to find a simple way of distinguishing between
> > > > interactive logins and jobs submitted via SGE
> > > > 
> > > > Thanks
> > > > MArtyn
> > > > 
> > > > ----------------------------------------------
> > > > Dr. Martyn D. Wheeler
> > > > Department of Chemistry
> > > > University of Leicester
> > > > University Road
> > > > Leicester, LE1 7RH, UK.
> > > > Tel (office): +44 (0)116 252 3985
> > > > Tel (lab):    +44 (0)116 252 2115
> > > > Fax:          +44 (0)116 252 3789
> > > > Email:        martyn.wheeler at le.ac.uk
> > > > http://www.le.ac.uk/chemistry/staff/mdw10.html
> > > >  
> > > > 
> > > >
> > > 
> ---------------------------------------------------------------------
> > > > To unsubscribe, e-mail:
> > > > users-unsubscribe at gridengine.sunsource.net
> > > > For additional commands, e-mail:
> > > > users-help at gridengine.sunsource.net
> > > > 
> > > > 
> > > 
> > > 
> > > 
> > > 		
> > > __________________________________ 
> > > Do you Yahoo!? 
> > > Yahoo! Mail - 250MB free storage. Do more. Manage less. 
> > > http://info.mail.yahoo.com/mail_250
> > > 
> > > 
> ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > > For additional commands, e-mail: 
> users-help at gridengine.sunsource.net
> > > 
> > > 
> > 
> > 
> ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> > 
> > 
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
> 
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list