[GE users] sge_security.c errors

Shannon V. Davidson svdavidson at swbell.net
Mon Nov 29 22:08:00 GMT 2004


If you are only interested in extending AFS tokens, I think there is a 
much simpler way to do it.  There is a little bit of information about 
this in the sge_conf(5) man page under set_token_cmd, pag_cmd, and 
token_extend_time.  I don't have any experience using these, but I 
believe there are those on this lists who use them.

In spite of what security.html says, the stuff in security/krb isn't 
really usable.  I developed this code many years ago (1997, I think) for 
a customer which required Kerberos security at their site. This was a 
full Kerberos implementation which used the Kerberos libraries for all 
communication between the daemons and clients. However, this  code was 
never put into production and has not been used at any production sites. 
 It was not fully tested and it has not been kept up-to-date with the 
many changes that have been put into Grid Engine since that time. The 
Kerberos support compiled into Grid Engine should be considered 
experimental.  There were several reasons for not finishing this 
implementation (e.g. time and money), but the main reason was the 
impracticality of supporting this version as a product back then (long 
before Grid Engine was open source) because of export restrictions on 
Kerberos itself and other practical considerations.  At that time, 
allowing the customer to compile the code on his own was simply not an 
option, because we didn't supply the source code to customers.

However, there is a more recent GSS-API Kerberos implementation that I 
used regularly in my Grid Engine 5.3 development and test environments 
and which is used full-time at least one production site which is 
running Grid Engine 5.3.  This implementation is different in that it is 
not a full Kerberos implementation but uses Kerberos to authenticate 
users submitting jobs and to forward user credentials with the job by 
calling security sub-programs at the appropriate times.  This version 
does not require recompiling Grid Engine.  It consists of some security 
modules which can be compiled separately and are called by Grid Engine 
to do authentication and to forward the Kerberos credentials.  The 
security sub-modules are called by client commands (e.g. qsub) and by 
the Grid Engine daemons (sge_qmaster, sge_execd) at the appropriate 
times to get and store credentials.  The Kerberos modules are used by 
Grid Engine when it is running in Kerberos mode (i.e. For GE 5.3, the 
$SGE_ROOT/default/common/product_mode file contains the string 
"sgeee-kerberos" or "sge-kerberos").  The source code for this 
implementation is located in the directory 
gridengine/source/security/gss.  The source code is not dependent on 
other Grid Engine components or libraries and can be compiled 
stand-alone.  Details on how to use this implementation can be found in 

If you need Kerberos to authenticate users who are submitting jobs to 
allow Grid Engine jobs to run with Kerberos credentials (which have been 
forwarded and are protected by encryption), then the GSS-API Kerberos 
implementation is the way to go.  Full authentication and encrypted 
communication via Kerberos between all Grid Engine clients (e.g. qmon, 
qstat) and deamons would require using the Kerberos code that you found, 
but I'm sure this would involve a significant amount of further testing 
and development. I have not attempted to build or test GSS under Grid 
Engine 6.x.  The last time I touched this stuff was about a year ago 
when I compiled it for a 5.3 environment.  Here's an email which 
describes what I did:

> The gss code that I'm referring to is in 
> gridengine/source/security/gss and is designed to compile standalone.  
> There is a separate aimk script in that directory.  You will probably 
> have to update KRB_HOME in the aimk file.  You should not compile Grid 
> Engine itself with the -gss or -kerberos flags.  Instead, take a look 
> at the instructions in 
> gridengine/source/security/gss/doc/gss_customer.html and 
> gridengine/source/security/gss/doc/gss.h for instuctions on compiling 
> and integrating the credential subprograms.
> Here's how I built them.  First, you want to create a security source 
> distribution.  This would typically be done so that the security 
> programs could be taken and installed and compiled at a customer site.
>     $ export SGE_ROOT=<a-valid-sge-distribution>
>     $ cd <your-ge-development-base-dir>/gridengine/source
>     $ ln -s scripts/distinst myinst
>     $ myinst -- sec                                 # this installs
>     the security sources in $SGE_ROOT/security
> Then you can compile and install them.
>     $ cd $SGE_ROOT/security
>     $ aimk -gss
>     $ aimk install
> I ran into 1 compile error and had to comment out the following line 
> in delete_cred.c.
>     #include "sge_stat.h"


Paul Mitchell wrote:

>Hello All,
>  After working with our local AFS guru, we decided to go the route of
>kerberos token renewal; so, following the instructions in
>gridengine/source/security/security.html, we've created the principals for
>the qmaster, schedd and execution hosts.  I've created a new keytab
>(/etc/grid.keytab) since we're running the grid executables as user
>"grid" and not root.
>Following the instructions in the implementation html:
>	gridengine/source/security/krb/doc/Implementation.html
>I performed a gridengine/source/aimk clean
>then a  gridengine/source/aimk -kerberos
>However, it blows out with:
>gcc -O3 -no-cpp-precomp -flat_namespace -Wall -Werror -Wstrict-prototypes
>-I../security/krb -I/vol2/tools/SW/krb5/darwin/include
>-I/vol2/tools/SW/krb5/darwin/include/gssapi -I/vol2/darwin/include
>-I/vol2/tools/SW/openssl-0.9.7d/darDCOMPILE_DC -D__SGE_NO_USERMAPPING__
>-I../security/sec -I../common -I../libs -I../libs/uti -I../libs/gdi
>-I../libs/japi -I..-I../libs/cull -I../libs/rmon -I../libs/comm
>-I../libs/comm/lists -I../libs/sched -I../libs/evc -I../libs/evm
>-I../libs/mir  -I../daemons/common -I../daemons/qmaster -I../daemons/execd
>-I../daemons/schedd -I../clients/common -I.  -dynamic -c ../liburity.c
>cc1: warnings being treated as errors
>../libs/gdi/sge_security.c: In function `store_sec_cred':
>../libs/gdi/sge_security.c:966: warning: implicit declaration of function
>../libs/gdi/sge_security.c:966: warning: format argument is not a pointer
>(arg 6)
>../libs/gdi/sge_security.c:974: parse error before "FAR"
>../libs/gdi/sge_security.c: In function `kerb_job':
>../libs/gdi/sge_security.c:1099: warning: format argument is not a pointer
>(arg 5)
>../libs/gdi/sge_security.c:1106: parse error before "FAR"
>../libs/gdi/sge_security.c: In function `tgt2cc':
>../libs/gdi/sge_security.c:1157: warning: assignment discards qualifiers
>from pointer target type
>../libs/gdi/sge_security.c:1162: warning: format argument is not a pointer
>(arg 4)
>../libs/gdi/sge_security.c:1173: parse error before "FAR"
>make: *** [sge_security.o] Error 1
>not done
>which is strange. Looking at sge_scutiy.c, the offending lines are:
>if ((rc = krb_encrypt_tgt_creds(tgt_creds, &outbuf))) {
>            request->host, request->commproc, request->id,
>         }
>it seems to be complaining about the error_messages(rc) if I'm reading
>this right?
>however, what's truely confusing is that there is no string "FAR" in the
>file - line 973-974 is:
> if (outbuf.length)
>    krb5_xfree(outbuf.data);
>Has anyone run into this before?
>Paul (heading back to reread the README.BUILD doument...)
>	Paul Mitchell
>	email: pmitchel at email.unc.edu
>	phone: (919) 962-9778
>	office: I have an office, room 14, Phillips Hall
>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>For additional commands, e-mail: users-help at gridengine.sunsource.net


Shannon V. Davidson <svdavidson at swbell.net>
Senior Software Engineer           Raytheon
636-479-7465 office        443-383-0331 fax

More information about the gridengine-users mailing list