[GE users] How to configure GE to send jobs to two clusters

Chris Dagdigian dag at sonsorol.org
Thu Apr 24 17:45:56 BST 2008

Hi Kevin,

As a general rule when I'm acting annoyingly vague on the list it's  
because I'm talking about organizations that I've visited as part of  
my day job. Without explicit permission from the people who invited me  
onsite I basically don't name names or divulge any real details.

"people" I've seen successfully doing neat things with qsub wrappers  

- Large group involved in weather modeling
- Hollywood types doing film rendering and production workflows with SGE
- Financial service / Oil & Gas firms doing all sorts of modeling

Can't get more specific, sorry!


On Apr 24, 2008, at 12:33 PM, Kevin Doman wrote:
> Hi Chris - Who are those "people"?
> My 600+ processors is quite busy and we have a qsub wrapper that  
> take would take a queue name, parse it, and add the -l option to it.  
> The submission time increase significantly with the number of jobs  
> submitted, but at least it works. For the nicer users, I just gave  
> them the direct access to the renamed qsub binary and all are  
> happy. :)
> On Thu, Apr 24, 2008 at 11:13 AM, Chris Dagdigian <dag at sonsorol.org>  
> wrote:
> The only people I've seen doing this in a really robust, production  
> environment are organizations that have wrapped and replaced the  
> standard qsub binary with a wrapper that has "local intelligence"  
> within it. The wrapper enforces some submission requirements, does  
> some other local stuff that makes sense and then makes the decision  
> "which" cluster gets the job submission.
> Otherwise qsub can be pointed at any cluster simply by initializing  
> the proper setup script and/or playing with path and ENV ports such  
> as SGE_QMASTER_PORT etc. Even then, you may want to wrap qsub if  
> only so the user gets back a "Your job has been submitted to CLUSTER  
> A" or some other sort of feedback regarding which system took their  
> request.
> Transfer queues can be used to move jobs between SGE clusters but  
> honestly that topic is something I've only encountered in  
> documentation and have never seen in a production environment.
> Same goes for qstat. If you just want to point qstat at an arbitrary  
> qmaster you just need to play games with ENV variables and/or the  
> settings.sh|csh scripts.
> If you want a single qstat command that shows output from both  
> clusters then you are back in custom wrapper land. Maybe a wrapper  
> that does an XML query to both systems and uses XSLT transformation  
> to print a plaintext report that is formatted to exactly suit your  
> needs. Otherwise, a web dashboard may be a more suitable interface  
> for monitoring multiple clusters.
> My $.02 of course!
> -Chris
> On Apr 24, 2008, at 12:04 PM, Kevin Doman wrote:
> Hello all - Count me in! I want something like this also. :)
> On Thu, Apr 24, 2008 at 10:24 AM, Esteban Freire <esfreire at cesga.es>  
> wrote:
> Hello all,
> We have two GE clusters configured, one installed locally and the  
> other one in an external machine running qmaster server. We would  
> like to know if it is possible configure our submitter host to  
> submit jobs to two different qmasters, I think we can get this  
> playing with the variable $SGE_CELL but I would appreciate some help.
> Other question in which we are interested is, can we also have a  
> qstat result for both qmasters?
> Thanks,
> Esteban

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list