[GE users] sanity check on idea to use project value to do fairshare among users with same uid

Andy Schwierskott andy.schwierskott at sun.com
Thu Nov 11 11:39:10 GMT 2004


your approach sounds reasonable for me. This even could make it possible to
raise the priority of a job within a project by using the -p Posix priority
switch, otherweise there is stricht FIFO of jobs within a project.


> Hi folks,
> I've got a web portal situation where many different scientific applications 
> are fronted by CGIs. Behind the scenes the CGIs are talking to Grid Engine to 
> load balance across apple and linux clusters.
> Because the jobs are coming from the web; they all run under the same UID and 
> Grid Engine just sees a ton of jobs being run by user "apache" or "www" or 
> whatever.
> Grid Engine can not effectively do fairshare-by-user when all it sees is many 
> jobs being submitted by the webserver.
> So we have the potential for one user out of many web portal users to grab a 
> huge percentage of cluster resources. Grid Engine has no way to sort this 
> out.
> There is no desire to provide commandline access to the cluster so there will 
> never be a per-user unique userid set up.
> What I've come up with was this:
> 1. Each web user gets a Grid Engine project created with their 
> portal-username. This is done automatically each time a new web portal user 
> is created.
> 2. The portal CGIs are modified to pass on "-P <username>" whenever they do a 
> qsub or qrsh
> 3. Adjust Grid Engine 5.3 or 6.0u1 policies such that "Project" is highly 
> weighted within the fairshare or sharetree policies
> The end result is that by hijacking the Grid Engine project mechanism I can 
> effectively get fairshare-by-user even when all the jobs are submitted with 
> the same UID.
> This appears to work and I can even now do per-user resource accounting 
> tracking with grid engine by doing:
> # Get per user usage stats:
> $ qacct -P <portal-username>
> Just wanted to throw this out there to see if anyone sees any nasty problems 
> that could come back to bite me. Anyone see any problems with Grid Engine if 
> used in a setting where for each user there is an associated project object?
> Regards,
> Chris


- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Andy Schwierskott           Tel: +49 (0)941 3075-200 (x60200)
N1 Grid Engine Engineering  Fax: +49 (0)941 3075-222 (x60222)
Sun Microsystems GmbH
Dr.-Leo-Ritter-Str. 7       mailto:andy.schwierskott at sun.com
D-93049 Regensburg          http://www.sun.com/gridware

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list