[GE users] Sourcing user's profile in jobs submitted with qrsh

dbarnes david.barnes at zenverge.com
Thu Jun 18 17:12:56 BST 2009

Hi Filipe,

Not everyone's usage model/choice, but we use "qrsh -V" to push the Users'
calling shell env. to the Execution Host.


-----Original Message-----
From: filbranden [mailto:lists.filbranden at idilia.com] 
Sent: Thursday, June 18, 2009 6:09 AM
To: users at gridengine.sunsource.net
Subject: [GE users] Sourcing user's profile in jobs submitted with qrsh


Unlike with qsub, jobs submitted with qrsh do not source the user's 
profile before running. I have a couple of issues that are created 
because of this behaviour:

* The "umask" settings are not respected, so users that usually have an 
umask of 002 will have their jobs run with an umask of 022, which will 
make the files not group-writable and this will create problems when 
other users in the same team will try to run jobs writing to the same 
directory trees.

* The "ulimit -c" (core file size) settings are not respected, many 
users set those to 0 in their profile files, so that programs that crash 
will not create huge core files, but when they launch grid jobs that 
crash they will often create core files in their home directories, 
eventually filling their disk.

* Environment variables in the profile files are not sourced, in 
particular users tend to create a ~/bin directory and add it to PATH, so 
that they don't need to specify the full path to the binary when they 
run commands from there, however that does not work when submitting jobs 
with qrsh.

I found a way to work around this issue, by creating a script that I 
call "profileval" that I installed in /usr/bin with this content:

--------- >8 ---------
#! /bin/bash -l
eval "$*"
--------- >8 ---------

And then I set this variable:


After doing that, my qrsh commands work as I expect, since they will be 
executed by a login shell (bash -l) which will source /etc/profile and 
~/.bash_profile (which often sources ~/.bashrc) and only then will 
"eval" the arguments, the same way qrsh without a wrapper seems to do.

This works fine for me as all my users have bash as their shells (this 
is a Linux platform deployment.)

(By the way, this script is also useful in cron jobs that suffer from 
the same issue. Often just adding "profileval" before the command is 
enough to set variables including PATH appropriately.)

My question is: Is there any side effects I am overlooking with this 
solution? In most cases I tested the behaviour of qrsh with and without 
the wrapper, and other than umask, ulimit and environment variables 
being there, everything including command line parsing seems to work the 
same way. Should I worry about other side effects such as having extra 
processes running and maybe signals not behaving the same way (for 
instance, problems killing jobs?).

And also, using the QRSH_WRAPPER variable is OK on a user-by-user basis, 
but if I want to activate this behaviour by default for all users, is 
there any SGE configuration variable I could use that would be used by 
qrsh even if the QRSH_WRAPPER variable is not set?

Is there any better way to solve this problem? I'm a grid newbie so I 
don't really know SGE very well, most of this I found with trial and 
error after reading about the QRSH_WRAPPER variable in one of the man 
pages, but it's possible that I'm looking at this the wrong way. I would 
appreciate it to hear from others that had the same issues and fixed 
them in different ways.



To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list