[GE users] combined/composite complexes

Olesen, Mark Mark.Olesen at arvinmeritor.com
Tue Apr 5 10:13:15 BST 2005


Servus,

There's nothing quite so nice as the "Aha-effect" that comes with the
removal of a sog. Denkfehler!

Cheers + Thanks,
/mark

Dr. Mark Olesen
Principal Engineer Thermofluids Analysis
ArvinMeritor Light Vehicle Systems
ArvinMeritor Emissions Technologies GmbH
Biberbachstr. 9
D-86154 Augsburg, GERMANY
tel: +49 (821) 4103 - 862
fax: +49 (821) 4103 - 7862
Mark.Olesen at ArvinMeritor.com

> -----Original Message-----
> From: Reuti [mailto:reuti at staff.uni-marburg.de]
> Sent: Tuesday, April 05, 2005 11:04 AM
> To: users at gridengine.sunsource.net
> Subject: RE: [GE users] combined/composite complexes
> 
> Moin moin,
> 
> yes I see, then I'd suggest to submit:
> 
> parallel jobs with -l c=1,a=1
> serial jobs with -l c=1,b=1
> parallel ones which may use serial licenses -l c=1
> 
> -pe mpi 6 -l A=1,C=1
> 
> wil give A=2,B=4,C=6 left, additional:
> 
> -pe mpi 6 -l C=1
> 
> will give A=2,B=4,C=0
> 
> As C is always requested, there should be no oversubscription by the limit
> of
> C=12 in your case. - Reuti
> 
> 
> Quoting "Olesen, Mark" <Mark.Olesen at arvinmeritor.com>:
> 
> > Hi Reuti,
> >
> > The problem is unfortunately that the "combined" licenses are actually
> used
> > for parallel jobs. I normally don't to make all/any of the "B" licenses
> > available for parallel jobs, since this prevents interactive use. Only
> > under
> > certain circumstances should the user request the "C" combination and
> block
> > some of the GUIs.
> >
> > We have 8 "A" licences and 4 "B" licenses.
> > Assuming that the first parallel job starts with "-pe mpi 6 -l A=1",
> then I
> > have 2 "A" licenses remaining and 4 "B" licenses. A second 6-cpu
> parallel
> > job would really need "-pe mpi 6 -l A=0.3333,B=0.6667" in order to
> start.
> >
> > I tried introducing the complex "C" and adjusted the availability of A,
> B &
> > C via an external load sensor that adjusts the complex_values for the
> > global
> > host. This, however, has two major problems:
> >   1) it relies on the license server and thus has the problems due to
> lag.
> >   2) oversubscription!  On an empty cluster, I have 8 "A" consumables
> >      and 12 "C" consumables. Mixed job requests (some with "-l A=1"
> >      and some with "-l C=1") are destined to failure.
> >
> > Perhaps there is some way of abusing a transfer queue to juggle the
> license
> > requests?
> >
> > /mark
> >
> > Dr. Mark Olesen
> > Principal Engineer Thermofluids Analysis
> > ArvinMeritor Light Vehicle Systems
> > ArvinMeritor Emissions Technologies GmbH
> > Biberbachstr. 9
> > D-86154 Augsburg, GERMANY
> > tel: +49 (821) 4103 - 862
> > fax: +49 (821) 4103 - 7862
> > Mark.Olesen at ArvinMeritor.com
> >
> > > -----Original Message-----
> > > From: Reuti [mailto:reuti at staff.uni-marburg.de]
> > > Sent: Tuesday, April 05, 2005 12:49 AM
> > > To: users at gridengine.sunsource.net
> > > Subject: Re: [GE users] combined/composite complexes
> > >
> > > Hi Mark,
> > >
> > > you already suggested a Complex C, which you can use for your parallel
> > > jobs.
> > > For serial runs, you have to request:
> > >
> > > -l C=1,B=1
> > >
> > > this will lower both resources.
> > >
> > > Cheers - Reuti
> > >
> > >
> > > Quoting "Olesen, Mark" <Mark.Olesen at arvinmeritor.com>:
> > >
> > > > Yet another license management problem :(
> > > >
> > > > Our CFD software has two different FlexLM "features" in use that are
> > > > tracked
> > > > within gridengine via complexes:
> > > >
> > > > Complex / Feature A
> > > >   * licenses for parallel calculations.
> > > > Complex / Feature B
> > > >   * licenses for interactive / serial / parallel use.
> > > >
> > > >
> > > > Generally, the system works quite well with this division. However,
> > > under
> > > > certain circumstances, it is desirable to allow a parallel job use
> some
> > > of
> > > > the licenses from complex B.
> > > >
> > > > The first idea was to invent a virtual license group that represents
> > the
> > > > combination of A and B:
> > > > 	Complex C = Complex A + Complex B
> > > >
> > > > Oversubscription is, however, a very obvious problem.
> > > >
> > > > What I need is a means of letting the gridengine know that it can
> use
> > > both
> > > > types of resources:
> > > >
> > > > For example,
> > > > 	qsub -pe mpi 10 -l A=1,[B=1]
> > > > could mean "use B as well if you run out of A"
> > > >
> > > > If I were to know a priori the exact number of licenses B that I
> needed
> > > to
> > > > use, I could write
> > > > 	qsub -pe mpi 10 -l A=0.8,B=0.2
> > > >
> > > > This, however, requires know the exact state of A & B when the job
> > > actually
> > > > starts. Otherwise the internal bookkeeping will be incorrect, since
> the
> > > > software always occupies all of feature A before it starts using
> > feature
> > > B.
> > > >
> > > > I'm hoping that someone has a solution or workaround in their bag-
> of-
> > > tricks
> > > > (Andy + Reuti perhaps?) for this problem.
> > > >
> > > > Regards,
> > > > /mark
> > > >
> > > >
> > > > Dr. Mark Olesen
> > > > Principal Engineer Thermofluids Analysis
> > > > ArvinMeritor Light Vehicle Systems
> > > > ArvinMeritor Emissions Technologies GmbH
> > > > Biberbachstr. 9
> > > > D-86154 Augsburg, GERMANY
> > > > tel: +49 (821) 4103 - 862
> > > > fax: +49 (821) 4103 - 7862
> > > > Mark.Olesen at ArvinMeritor.com
> > > >
> > > > --------------------------------------------------------------------
> -
> > > > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > > > For additional commands, e-mail: users-help at gridengine.sunsource.net
> > > >
> > >
> > >
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > > For additional commands, e-mail: users-help at gridengine.sunsource.net
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> >
> 
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list