[GE users] sanity check on idea to use project value to do fairshare among users with same uid
andy.schwierskott at sun.com
Thu Nov 11 11:39:10 GMT 2004
your approach sounds reasonable for me. This even could make it possible to
raise the priority of a job within a project by using the -p Posix priority
switch, otherweise there is stricht FIFO of jobs within a project.
> Hi folks,
> I've got a web portal situation where many different scientific applications
> are fronted by CGIs. Behind the scenes the CGIs are talking to Grid Engine to
> load balance across apple and linux clusters.
> Because the jobs are coming from the web; they all run under the same UID and
> Grid Engine just sees a ton of jobs being run by user "apache" or "www" or
> Grid Engine can not effectively do fairshare-by-user when all it sees is many
> jobs being submitted by the webserver.
> So we have the potential for one user out of many web portal users to grab a
> huge percentage of cluster resources. Grid Engine has no way to sort this
> There is no desire to provide commandline access to the cluster so there will
> never be a per-user unique userid set up.
> What I've come up with was this:
> 1. Each web user gets a Grid Engine project created with their
> portal-username. This is done automatically each time a new web portal user
> is created.
> 2. The portal CGIs are modified to pass on "-P <username>" whenever they do a
> qsub or qrsh
> 3. Adjust Grid Engine 5.3 or 6.0u1 policies such that "Project" is highly
> weighted within the fairshare or sharetree policies
> The end result is that by hijacking the Grid Engine project mechanism I can
> effectively get fairshare-by-user even when all the jobs are submitted with
> the same UID.
> This appears to work and I can even now do per-user resource accounting
> tracking with grid engine by doing:
> # Get per user usage stats:
> $ qacct -P <portal-username>
> Just wanted to throw this out there to see if anyone sees any nasty problems
> that could come back to bite me. Anyone see any problems with Grid Engine if
> used in a setting where for each user there is an associated project object?
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Andy Schwierskott Tel: +49 (0)941 3075-200 (x60200)
N1 Grid Engine Engineering Fax: +49 (0)941 3075-222 (x60222)
Sun Microsystems GmbH
Dr.-Leo-Ritter-Str. 7 mailto:andy.schwierskott at sun.com
D-93049 Regensburg http://www.sun.com/gridware
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net
More information about the gridengine-users