[GE globus] Re: VDT modifications to Globus 2 SEG

Jeff Porter RJPorter at lbl.gov
Wed Jul 16 03:03:42 BST 2008



Yes, I believe that is true of all 3 versions:  LeSC, Graham's modified version, and the one currently in VDT.  There also appears to be differences between the different jobmanagers (including condor) in how they handle jobtype=single and count>1.     It would be useful to resolve such differences is in a more coordinated effort such as,

http://dev.globus.org/wiki/Incubator/LRMA

There has been some initial discussions about getting this project up and running.

Jeff


> 
> Hello,
> 
> I believe that the LeSC version of the jobmanager-sge (sge.pm) has a
> problem in case of jobtype=multiple.
> It launches $count jobs on one node.  We would expect the behavior of
> multiple jobs in jobmanager-sge should be the same with the behavior
> in jobmanager-pbs, which allocates $count CPUs for $count jobs.
> We have confirmed that the Globus team (Stu Martin) had the same
> opinion.
> 
> Thanks,
> 
> --
> Yoshio Tanaka (yoshio.tanaka at aist.go.jp)
> http://ninf.apgrid.org/
> http://www.apgridpma.org/
> 
> 
> From: Jeff Porter <rjporter at lbl.gov>
> Subject: Re: VDT modifications to Globus 2 SEG
> Date: Fri, 11 Jul 2008 13:54:17 -0700
> Message-ID: <4877C879.3090906 at lbl.gov>
> 
> > 
> > So I just did the same and, after a couple of definition changes, 
> also 
> > had success.  It looks like this script from Graham/Vladimir is 
> > minimally changed from the one currently on the LeSC web while 
> the VDT 
> > version differs quite a bit.  I don't know when these two scripts 
> > diverged but I had been told that the VDT one had been modified 
> to work 
> > correctly with GT2 - perhaps the LeSC version changed as well but 
> more 
> > intelligently ;-) .  I think Graham had forwarded me this script 
> earlier 
> > but I hadn't made the comparison before.
> > 
> > In the end, it looks like Graham/Vladimir version is a good 
> candidate 
> > for replacing what's currently in the VDT - it removes the 
> ".real" hack 
> > and does a better test for job-state.   It does need a patch for 
> an OSG 
> > definition and I'd like to run some tests using some currently 
> working 
> > production scripts, but after that I'll check with Alain about 
> updating 
> > the VDT.
> > 
> > thanks,
> > Jeff
> > 
> > 
> > On 7/11/08 7:41 AM, John Sanabria wrote:
> > > Hi,
> > >
> > > I have copied Vladimir's sge.pm file to
> > >
> > > -> /opt/vdt/./globus/lib/perl/Globus/GRAM/JobManager/sge.pm
> > > -> /opt/vdt/./globus/setup/globus/sge.pm
> > >
> > > Previous to this change I had this situation. When I submitted 
> jobs 
> > > using globus-job-submit and check for its status I always got a 
> > > PENDING message although the job was running.
> > >
> > > Now, when I submit a job and check for its status I got the 
> correct 
> > > answer "ACTIVE".
> > >
> > > Thanks Vladimir for your always useful support and Blair let me 
> know 
> > > how your issue goes.
> > >
> > > Regards and happy weekend ;-)
> > >
> > > Vladimir Mencl wrote:
> > >> On 07/11/2008 11:35 AM, Jeff Porter wrote:
> > >>>
> > >>> Hi Vladimir,
> > >>>
> > >>> Could you or Graham forward me your sge.pm file?    I 
> remember 
> > >>> thinking the exact same thing as you (that this bug is 
> obsolete) and 
> > >>> had tried just removing ".real" part. That ended up not 
> working 
> > >>> correctly.  I think the problem was that the globus job 
> manager's 
> > >>> stdout/err would overwrite the job's stdout/err (this is from 
> my 
> > >>> memory - I've not had the chance to look at this today).  I 
> think 
> > >>> this problem is a GT2-SGE mismatch and doesn't come into play 
> with 
> > >>> GT4.  So if you're only running GT4, then that additional 
> code is 
> > >>> irrelevant and I also skip it in the VDT's sge.pm.
> > >>> I do agree that your "-c" suggestion would be a good 
> additional test.
> > >>
> > >> Hi Jeff,
> > >>
> > >> Please find attached my current sge.pm.  It's based on the one 
> I got 
> > >> from Graham Jenkins, and differs only in minor customizations 
> to the 
> > >> local environment.
> > >>
> > >> However, I am only running GT4 (and I expect so does Graham), 
> so the 
> > >> file may be irrelevant for a discussion on whether a problem 
> with GT2 
> > >> has been resolved....
> > >>
> > >> Cheers,
> > >> Vladimir
> > >>
> > >>>
> > >>> thanks,
> > >>> Jeff
> > >>>
> > >>>
> > >>> On 7/10/08 3:19 PM, Vladimir Mencl wrote:
> > >>>> On 07/11/2008 02:06 AM, Blair Bethwaite wrote:
> > >>>>> On Thu, Jul 10, 2008 at 11:45 PM, Yoshio Tanaka
> > >>>>> <yoshio.tanaka at aist.go.jp> wrote:
> > >>>>>> Which version of the LeSC Globus-SGE adapter is included 
> in VDT?
> > >>>>>
> > >>>>> Good point Yoshio, I should have looked at that to begin 
> with! The
> > >>>>> headers in sge.pm on Komolongma say:
> > >>>>> # Marko Krznaric
> > >>>>> # London eScience Centre
> > >>>>> # June 2003
> > >>>>> #
> > >>>>> # Contributions by David McBride
> > >>>>> # London eScience Centre
> > >>>>> # Oct 2003
> > >>>>> #
> > >>>>> # Contributions by Jeff Porter
> > >>>>> # Lawrence Berkeley National Laboratory
> > >>>>> # Sept 2007
> > >>>>
> > >>>> Hi,
> > >>>>
> > >>>> I'm also running an SGE cluster with GT4, and I don't get 
> this 
> > >>>> problem. I'm using a sge.pm I got from Graham Jenkins, and 
> it does 
> > >>>> not have the code fragment which appends ".real" to whatever 
> > >>>> standard output and standard error are (lines 372-373).
> > >>>>
> > >>>> The comment just above these lines refers to a streaming 
> issue in 
> > >>>> Globus: bug #1288, 
> > >>>> http://bugzilla.globus.org/bugzilla/show_bug.cgi?id=1288
> > >>>> The bug is _ancient_ - dates from 2003, and is marked as 
> resolved.> >>>> I would expect it's safe to commented these lines out.
> > >>>>
> > >>>> Looking at the code itself, the code does not take into 
> account 
> > >>>> that stdout and stderr could be /dev/null, and should not 
> change 
> > >>>> the file name in that case.
> > >>>>
> > >>>> In the "if" condition at the end of the line, it does check 
> for a 
> > >>>> block special file (-b), but it should also check for a 
> character 
> > >>>> special file with -c:
> > >>>>
> > >>>>  print (....) if (! -b $description->stdout() && ! -c 
> > >>>> $description->stdout() );
> > >>>>                                               
> > >>>> ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
> > >>>> and similarly for stderr.
> > >>>>
> > >>>>
> > >>>> I would thus recommend either fixing the code to also check 
> for 
> > >>>> character special files, or commenting these lines out 
> completely.> >>>>
> > >>>>
> > >>>> Cheers,
> > >>>> Vladimir
> > >>>>
> > >>>>
> > >>
> > >>
> > >
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: globus-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: globus-help at gridengine.sunsource.net




More information about the gridengine-users mailing list