[GE users] AFS Support in SGE6.1

Duc Bao Ta ta at physik.uni-bonn.de
Thu May 10 16:12:35 BST 2007

    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]


this might be interesting for people who use AFS/Kerberos5 with SGE. 
Looking through the emails in the mailing list quite alot of solutions 
(and quite old ones) are around. I found something that worked for me 
now. I used SGE5 with GSS support and modified code to get an AFS token 
with the ticket BEFORE the "job output file writing" starts. Since the 
security modules of latest version (v6) does not seem to compile with 
the debian etch (even with all the code modifications that are around), 
I came up with this solution:

I downloaded the binaries (V6.1beta) and installed sge with 
./inst_qmaster -afs to activate the AFS support.

The scripts I need now are, get_token_cmd, set_token_cmd, pag and some 
helper python scripts (some comments are for debug only):

#--- script get_token_cmd:
#--- convert ticket to base64
/opt/sge/util/base64.convert.py $KRB5CCNAME $ticketfile
#--- print out ticket
cat $ticketfile
rm $ticketfile

#--- script set_token_cmd:
jobid=`echo $PWD | awk -F / '{ print $NF}'`
#exec 1>/tmp/set_token_cmd.$jobid 2>&1
#echo --- set_token_cmd
#--- in case KRB5CCNAME was not set by pag script
if [ -z $KRB5CCNAME ];
if [ -e $ticketname.txt ]
        rm $ticketname.txt
#--- receive the ticket
while read line
        echo $line >> $ticketname.txt
#echo $1 $2 $3 $4 $5
#--- restore the ticket
/opt/sge/util/base64.reconvert.py $ticketname.txt $ticketname
rm $ticketname.txt
export KRB5CCNAME=$ticketname
#klist -c $ticketname
#--- get AFS-token
chmod 600 $ticketname
chown $1:$1 $ticketname

#--- script pag:
export KRB5CCNAME=/tmp/ticket.`echo $PWD | awk -F / '{ print $NF}'`
/usr/bin/pagsh -c "$2"

#--- script base64.convert.py
import base64
import sys

#--- script base64.reconvert.py
import base64
import sys

For the set_token_cmd and pag you have to set the configuration to point 
to the files (set_token_cmd and pagsh in the cluster configuration), the 
get_token_cmd MUST be in $SGE_ROOT/util (where can I change that? where 
have all the text-configuration files gone?).

Here is how I think it works:
When you enter qsub, the get_token_cmd is executed which expects the 
"token" to be printed to stdout (if something is on stderr, the 
submission is aborted). When the job starts a new pag-shell is created 
by executing pag. Then set_token_cmd receives the "token" from stdin and 
the job is started.

Now I do not send the AFS-token, but the krb5-ticket coded as BASE64 
(the python scripts are doing the conversion, some characters get lost 
when coding with e.g. uuencode or sending the 'binary'). The pag script 
has to set the KRB5CCNAME variable (/tmp/ticket.**jobId**, I use pwd 
since the program seems to be in the directory of the stored script, 
which is [...]/**jobId**, I haven't found variables that contain the job 
number or task-id number) which stays valid for the set_token_cmd and 
the job itself (set_token_cmd and the job seem to be children of the pag 
script), otherwise the job does not know which ticket was used to get 
the AFS-token. set_token_cmd restores the original krb5-ticket and gets 
an AFS-token. After all that the job starts.

Using this I don't have to create extra keytabs for every user and the 
authentification is done by trying to get a token from the ticket that 
was sent at the time of submission.

I only wonder, where the "token" (i.e. my converted ticket) is stored 
and if it is securely transfered (maybe using CSP helps)?

Cheers Duc

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list