[GE users] Globus or not to Globus?

Ignacio Martín Llorente llorente at dacya.ucm.es
Wed Jan 11 13:57:04 GMT 2006


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Dear Stefan,

GridWay (http://www.gridway.org) is a light-weight meta-scheduler that 
performs job execution management and resource brokering. It allows 
unattended, reliable, and efficient execution of jobs, array jobs, or 
complex jobs on heterogeneous, dynamic and loosely-coupled Grids. 
GridWay performs all the job scheduling and submission steps 
transparently to the end user and adapts job execution to changing Grid 
conditions by providing fault recovery mechanisms, dynamic scheduling, 
migration on-request and opportunistic migration.

GridWay uses the Globus GRAM interface on computation resources. It is 
able to simultaneously use both pre-WS and WS GRAM services.

The GridWay command line interface is similar to that found on Unix and 
resource management systems such as PBS or SGE. It allows users to 
submit, kill, migrate, monitor and synchronize jobs. The framework 
automatically detects when a job has failed and allows the user to 
abort, retry or migrate it to a new machine. It also includes DRMAA GGF 
standard support to develop distributed applications.

I answer your questions in the first email:

 >Has anyone any experience with GT4 and SGE6? How difficult is it to 
set up?

GridWay is fully functional on GT4. In our research lab, GT4 and GridWay 
are used to harness SGE6 clusters.

 > Is there any added complexity for the users to submit jobs through 
globus compared to the regular 'qsub'.

No, please see Command Reference at http://www.gridway.org/commands.php 
and screenshots at http://www.gridway.org/screenshots.php

 > Does the user have to make an xml file to transfer files with gridftp 
or is there a simpler way?

No, input/output files and streams are specified in a job template file 
(http://www.gridway.org/jobtemplate.php) and transparently managed by 
GridWay.


Please, do not hesitate to contact us if you need any help related to 
GridWay or the configuration of GT4 for SGE6. The new release of GridWay 
(http://www.gridway.org/featureplan.php, version 5.0, is scheduled for 
April 2006. Check current and new release functionality in our 
Metascheduling Functionality Point Checklist at 
http://www.gridway.org/functionality.php

Best regards,

Ignacio

Andreas Haas escribió:
> Hi Stefan,
> 
> then I believe you got two options
> 
> (1) If ldap problems refers to inconsistent user name schemes you would
>     have to compile optional user mapping. Unfortunately that would
>     require you to use 5.3 Grid Engine since user mapping compile-time
>     option still is not yet fixed 6.0 source tree. To deal with the
>     remaining nfs server problem is doable based on
> 
>          http://gridengine.sunsource.net/howto/nfsreduce.html
> 
>     and solves the problem to the extend nfs access done by Grid Engine is
>     concerned.
> 
> (2) Rely on Globus functionality to deal with nfs and ldap problems. Since
>     I'm Globus-illiterate I can not tell you the drawbacks use of Globus
>     would have. Possibly GridWay (uses Globus) could help
> 
>          http://www.gridway.org/
> 
>     you. Though tith GridWay I either have no experience but I once saw a
>     good presentation and my impression was that it is usable. I cc
>     Ignacio Martin Llorente from GridWay project.
> 
> Regards,
> Andreas
> 
> On Wed, 11 Jan 2006 Stefan.O.Nordlander at astrazeneca.com wrote:
> 
> 
>>Hi Stephan and thanks for your reply,
>>
>>could you please explain that a bit further? Why would that be a problem? I
>>understand the benefits of Globus (authentication and file staging etc.) but
>>that would be the next step. If we already had a common ldap and nfs over
>>resonable bandwidth how would one go about joining three clusters? Would the
>>$SGEHOME have to be exported over NFS to all sites? And I guess that all
>>hell would break loose if that connection would be lost?
>>
>>Thanks,
>>/Stefan
>>
>>
>>
>>>-----Original Message-----
>>>From: Stephan Grell - Sun Germany - SSG - Software Engineer
>>>[mailto:stephan.grell at sun.com]
>>>Sent: den 11 januari 2006 09:46
>>>To: users at gridengine.sunsource.net
>>>Subject: Re: [GE users] Globus or not to Globus?
>>>
>>>
>>>Hi Stefan,
>>>
>>>the easiest way would be to join the three grids into one.
>>>But I assume
>>>that is not possible?
>>>If you could do that, you would not need Globus.
>>>
>>>Cheers,
>>>Stephan
>>>
>>>Stefan.O.Nordlander at astrazeneca.com wrote On 01/10/06 14:12,:
>>>
>>>
>>>>Hi,
>>>>
>>>>I'm wondering what the simplest possible way would be to
>>>
>>>connect three
>>>
>>>>clusters using SGE. Nevermind user accounts or shared home
>>>
>>>directories etc.
>>>
>>>>Would it be possible to somehow make the other SGE
>>>
>>>installations aware of
>>>
>>>>each other.
>>>>
>>>>Basically what we need is a way to send jobs from any
>>>
>>>workstation on site A
>>>
>>>>to the cluster at site A, B or C. It would be ok for the
>>>
>>>user to copy the
>>>
>>>>files manually. And the authentication is done through ldap
>>>
>>>on all sites.
>>>
>>>>
>>>>Globus seems complicated at a first look. Certificates,
>>>
>>>proxy, grid-ftp and
>>>
>>>>the lot. Still I understand the benefits of having a 'grid'.
>>>
>>>Has anyone any
>>>
>>>>experience with GT4 and SGE6? How difficult is it to set up?
>>>>
>>>>Is there any added complexity for the users to submit jobs
>>>
>>>through globus
>>>
>>>>compared to the regular 'qsub'. Does the user have to make
>>>
>>>an xml file to
>>>
>>>>transfer files with gridftp or is there a simpler way?
>>>>
>>>>
>>>>Thanks,
>>>>/Stefan
>>>>
>>>>---------------------------------------------------------------------
>>>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>>
>>>>
>>>>
>>>
>>>---------------------------------------------------------------------
>>>To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
>>>For additional commands, e-mail: users-help at gridengine.sunsource.net
>>>
>>
>>
> 

-- 
Ignacio Martin Llorente
http://asds.dacya.ucm.es/nacho
----------------------------------------------------------

GridWay, Meta-scheduling Technologies for the Grid! http://www.gridway.org

----------------------------------------------------------
You are invited to visit our research web page http://asds.dacya.ucm.es
----------------------------------------------------------

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list