[GE users] AFS and SGE
Wolfgang.Friebel at desy.de
Tue Feb 8 11:12:53 GMT 2005
On Tue, 8 Feb 2005, Marco Donauer wrote:
> Wolfgang Friebel (also on the list) is using SGE with AFS. Perhaps he can
> help you?
Maybe yes. I was already trying to put something together when your mail
arrived. Just to get some information to the list I can give a short overview
what we are doing to have SGE with AFS.
1) Install the qmaster and exec hosts with the additional switch -afs
(e.g. install_qmaster -afs or inst_sge -m -afs)
Then you should see AFS related info in the output of qconf -sconf
similar to (assuming that SGE_ROOT is /usr/SGE):
2) At qsub time a script /usr/SGE/util/get_token_cmd
is called that can be used to fetch the AFS token from memory and store it
together with the job information for later use. For the procedure described
below it is sufficient to check that the user is authenticated.
At job start time another script /usr/SGE/util/set_token_cmd is called
that will restore or create an AFS token for a given user. The procedure
proposed originally did retrieve the AFS token stored with get_token_cmd
and then by decoding and reassembling the token with an increased lifetime
a new AFS token was stored in the process environment (memory).
The procedure currently used by us is different and is based on Kerberos5:
The SGE daemons are running as root and do have access to a Kerberos5 key file
where a key for a special privileged kerberos principal is stored. (Another
user that has access to that key file could be used as well)
By means of this key file a ticket for that principal is obtained and a
remote Kerberos5 aware server is contacted. That server verifies the access
rights of the client (which is the set_token_cmd script). That is done by
verifying the validity of the K5 ticket obtained by the client.
The server itself has access to another keytab file where the key for a k5
principal with admin rights is stored. The server gets this way a K5 admin
ticket and with that ticket the Kerberos5 KDC can be contacted and a key file
for a given user is created. That key file is used to create a
credentials cache which is transferred from the server to
the client in a secure way. Then on the client this cache provides the
Kerberos ticket for the user. If Heimdal-Kerberos5 is used, then an
AFS token can be generated simultaneously, otherwise (MIT) aklog can be called
to get an AFS token from the K5 ticket.
It is important to get the AFS token in a pagsh environment. Otherwise
nasty side effects will occur.
3) For the communication between the qmaster and the shadow masters we do use
AFS to share common files. As there are other methods to share these files
(Berkeley DB spooling, other Filesystems like NFS) this step is optional. We
have been using IP based ACL's in AFS to select the proper access privileges
from the servers to the shared files like follows:
pts creategroup sgeserver
pts createuser <ip_adr of qmaster> #qmaster
pts createuser <ip_adr of shadow master> #shadow1
pts adduser <ip_adr of qmaster> sgeserver
pts adduser <ip_adr of shadow master> sgeserver
Currently our get_token_cmd and set_token_cmd scripts are still highly
site specific and cannot easily be distributed. We could make available
the code for the kerberos5 aware server and client that was described above.
>From these programs the get_token_cmd and set_token_cmd scripts
could be easily derived. We will probably have such scripts ready for
distribution by end of April.
The K5 client and server are pure perl implementations based on Authen::SASL. I
have put the relevant files to our anonymous ftp server:
Authen-SASL-2.08.tar.gz (also on CPAN)
Authen-SASL-Cyrus-0.11.tar.gz (the version on CPAN is different and does
not support server side K5)
If you need more information please do not hesitate to contact me again
Wolfgang Friebel Deutsches Elektronen-Synchrotron DESY
Phone/Fax: +49 33762 77372/216 Platanenallee 6
Mail: Wolfgang.Friebel AT desy.de D-15738 Zeuthen Germany
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users