[GE users] combined/composite complexes

Reuti reuti at staff.uni-marburg.de
Tue Apr 5 10:03:47 BST 2005


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Moin moin,

yes I see, then I'd suggest to submit:

parallel jobs with -l c=1,a=1
serial jobs with -l c=1,b=1
parallel ones which may use serial licenses -l c=1

-pe mpi 6 -l A=1,C=1

wil give A=2,B=4,C=6 left, additional:

-pe mpi 6 -l C=1

will give A=2,B=4,C=0

As C is always requested, there should be no oversubscription by the limit of 
C=12 in your case. - Reuti


Quoting "Olesen, Mark" <Mark.Olesen at arvinmeritor.com>:

> Hi Reuti,
> 
> The problem is unfortunately that the "combined" licenses are actually used
> for parallel jobs. I normally don't to make all/any of the "B" licenses
> available for parallel jobs, since this prevents interactive use. Only
> under
> certain circumstances should the user request the "C" combination and block
> some of the GUIs.
> 
> We have 8 "A" licences and 4 "B" licenses.
> Assuming that the first parallel job starts with "-pe mpi 6 -l A=1", then I
> have 2 "A" licenses remaining and 4 "B" licenses. A second 6-cpu parallel
> job would really need "-pe mpi 6 -l A=0.3333,B=0.6667" in order to start.
> 
> I tried introducing the complex "C" and adjusted the availability of A, B &
> C via an external load sensor that adjusts the complex_values for the
> global
> host. This, however, has two major problems:
>   1) it relies on the license server and thus has the problems due to lag.
>   2) oversubscription!  On an empty cluster, I have 8 "A" consumables
>      and 12 "C" consumables. Mixed job requests (some with "-l A=1"
>      and some with "-l C=1") are destined to failure.
> 
> Perhaps there is some way of abusing a transfer queue to juggle the license
> requests?
> 
> /mark
> 
> Dr. Mark Olesen
> Principal Engineer Thermofluids Analysis
> ArvinMeritor Light Vehicle Systems
> ArvinMeritor Emissions Technologies GmbH
> Biberbachstr. 9
> D-86154 Augsburg, GERMANY
> tel: +49 (821) 4103 - 862
> fax: +49 (821) 4103 - 7862
> Mark.Olesen at ArvinMeritor.com
> 
> > -----Original Message-----
> > From: Reuti [mailto:reuti at staff.uni-marburg.de]
> > Sent: Tuesday, April 05, 2005 12:49 AM
> > To: users at gridengine.sunsource.net
> > Subject: Re: [GE users] combined/composite complexes
> > 
> > Hi Mark,
> > 
> > you already suggested a Complex C, which you can use for your parallel
> > jobs.
> > For serial runs, you have to request:
> > 
> > -l C=1,B=1
> > 
> > this will lower both resources.
> > 
> > Cheers - Reuti
> > 
> > 
> > Quoting "Olesen, Mark" <Mark.Olesen at arvinmeritor.com>:
> > 
> > > Yet another license management problem :(
> > >
> > > Our CFD software has two different FlexLM "features" in use that are
> > > tracked
> > > within gridengine via complexes:
> > >
> > > Complex / Feature A
> > >   * licenses for parallel calculations.
> > > Complex / Feature B
> > >   * licenses for interactive / serial / parallel use.
> > >
> > >
> > > Generally, the system works quite well with this division. However,
> > under
> > > certain circumstances, it is desirable to allow a parallel job use some
> > of
> > > the licenses from complex B.
> > >
> > > The first idea was to invent a virtual license group that represents
> the
> > > combination of A and B:
> > > 	Complex C = Complex A + Complex B
> > >
> > > Oversubscription is, however, a very obvious problem.
> > >
> > > What I need is a means of letting the gridengine know that it can use
> > both
> > > types of resources:
> > >
> > > For example,
> > > 	qsub -pe mpi 10 -l A=1,[B=1]
> > > could mean "use B as well if you run out of A"
> > >
> > > If I were to know a priori the exact number of licenses B that I needed
> > to
> > > use, I could write
> > > 	qsub -pe mpi 10 -l A=0.8,B=0.2
> > >
> > > This, however, requires know the exact state of A & B when the job
> > actually
> > > starts. Otherwise the internal bookkeeping will be incorrect, since the
> > > software always occupies all of feature A before it starts using
> feature
> > B.
> > >
> > > I'm hoping that someone has a solution or workaround in their bag-of-
> > tricks
> > > (Andy + Reuti perhaps?) for this problem.
> > >
> > > Regards,
> > > /mark
> > >
> > >
> > > Dr. Mark Olesen
> > > Principal Engineer Thermofluids Analysis
> > > ArvinMeritor Light Vehicle Systems
> > > ArvinMeritor Emissions Technologies GmbH
> > > Biberbachstr. 9
> > > D-86154 Augsburg, GERMANY
> > > tel: +49 (821) 4103 - 862
> > > fax: +49 (821) 4103 - 7862
> > > Mark.Olesen at ArvinMeritor.com
> > >
> > > ---------------------------------------------------------------------
> > > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > > For additional commands, e-mail: users-help at gridengine.sunsource.net
> > >
> > 
> > 
> > 
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> > For additional commands, e-mail: users-help at gridengine.sunsource.net
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
> 



---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list