[GE users] How to configure GE to send jobs to two clusters

Kevin Doman kdoman07 at gmail.com
Thu Apr 24 17:33:29 BST 2008


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Hi Chris - Who are those "people"?
My 600+ processors is quite busy and we have a qsub wrapper that take would
take a queue name, parse it, and add the -l option to it. The submission
time increase significantly with the number of jobs submitted, but at least
it works. For the nicer users, I just gave them the direct access to the
renamed qsub binary and all are happy. :)



On Thu, Apr 24, 2008 at 11:13 AM, Chris Dagdigian <dag at sonsorol.org> wrote:

>
> The only people I've seen doing this in a really robust, production
> environment are organizations that have wrapped and replaced the standard
> qsub binary with a wrapper that has "local intelligence" within it. The
> wrapper enforces some submission requirements, does some other local stuff
> that makes sense and then makes the decision "which" cluster gets the job
> submission.
>
> Otherwise qsub can be pointed at any cluster simply by initializing the
> proper setup script and/or playing with path and ENV ports such as
> SGE_QMASTER_PORT etc. Even then, you may want to wrap qsub if only so the
> user gets back a "Your job has been submitted to CLUSTER A" or some other
> sort of feedback regarding which system took their request.
>
> Transfer queues can be used to move jobs between SGE clusters but honestly
> that topic is something I've only encountered in documentation and have
> never seen in a production environment.
>
> Same goes for qstat. If you just want to point qstat at an arbitrary
> qmaster you just need to play games with ENV variables and/or the
> settings.sh|csh scripts.
>
> If you want a single qstat command that shows output from both clusters
> then you are back in custom wrapper land. Maybe a wrapper that does an XML
> query to both systems and uses XSLT transformation to print a plaintext
> report that is formatted to exactly suit your needs. Otherwise, a web
> dashboard may be a more suitable interface for monitoring multiple clusters.
>
> My $.02 of course!
>
> -Chris
>
>
>
>
> On Apr 24, 2008, at 12:04 PM, Kevin Doman wrote:
>
>> Hello all - Count me in! I want something like this also. :)
>>
>>
>> On Thu, Apr 24, 2008 at 10:24 AM, Esteban Freire <esfreire at cesga.es>
>> wrote:
>> Hello all,
>>
>> We have two GE clusters configured, one installed locally and the other
>> one in an external machine running qmaster server. We would like to know if
>> it is possible configure our submitter host to submit jobs to two different
>> qmasters, I think we can get this playing with the variable $SGE_CELL but I
>> would appreciate some help.
>>
>> Other question in which we are interested is, can we also have a qstat
>> result for both qmasters?
>>
>>
>> Thanks,
>> Esteban
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>> For additional commands, e-mail: users-help at gridengine.sunsource.net
>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>



More information about the gridengine-users mailing list