IZ285: allow parallel job allocation scheme be specified at submission time

[andreas]

[Imported from gridengine issuezilla ​http://gridengine.sunsource.net/issues/show_bug.cgi?id=285]

        Issue #:      285              Platform:     All           Reporter: andreas (andreas)
       Component:     gridengine          OS:        All
     Subcomponent:    clients          Version:      5.3              CC:
                                                                             [_] jmarshall
                                                                             [_] Remove selected CCs
        Status:       NEW              Priority:     P4
      Resolution:                     Issue type:    ENHANCEMENT
                                   Target milestone: ---
      Assigned to:    roland (roland)
      QA Contact:     roland
          URL:
       * Summary:     allow parallel job allocation scheme be specified at submission time
   Status whiteboard:
      Attachments:

     Issue 285 blocks:
   Votes for issue 285:


   Opened: Mon Jun 10 05:14:00 -0700 2002 
------------------------


DESCRIPTION:
The switch "-pe <pe-name> <slots-range>" requires
the admin to prepare a
parallel environment (PE) for each allocation rule
to be used within a cluster.
Depending on the profile of the parallel jobs
typically used at a site it might
be cumbersome to manage all the different PEs and
to have the user understand the meaning of each
PE.

WORKAROUND:
Create one PE for each allocation rule in use.

HOW TO FIX:
It would be possible to have the user directly
specify the
allocation rule to be used for a parallel job
instead of
having the admin to prepare PEs and the user
decide for
those prepared PEs. A new job submission switch

    "-pe_alloc <allocation_rule> <slot-range>"

could be introduced to support this. The new
switch needs
two option arguments:

1. <allocation_rule>
   Specifying an allocation rule with the
-pe_alloc switch
   is semantic-wise a synonynous for selecting a
PE providing
   this allocation rule

2. <slot-range>
   Has the same meaning as the "slot-range" option
argument with
   the existing -pe switch.

Then the user could directly control allocation
scheme e.g.
"-pe_alloc 2 8" or "-pe_alloc 4 64,48,32,16".

For a parallel job either "-pe <pe-name>
<slot-range>"
or "-pe_alloc <allocation_rule> <slot-range>" may
be specified.

   ------- Additional comments from andreas Wed Apr 16 02:36:06 -0700 2003 -------
The draft sketched under "HOW TO FIX" is not sufficient.
It must be reworked to allow user defined allocation
rules be used in a consistent combination with other features
of the PE object.

These are
* administrator controlled PE start/stop procedure
* administrator controlled access to the PE
* administrator controlled maxmimum amount of slots for a certain
  group of parallel jobs
* administrators decision whether for those parallel jobs
  the tight/loose integration must be used (!)

To allow this the draft should be reworked to allow jobs be rejected
at submission time in case

(1) the job requests an allocation rule without also refering to an
administrator provided PE object. By doing so one ensures e.g. that it
is specified whehter tight/loose integration must be assumed for the job.

(2) the PE object refered by the users allocation rule request is not
foreseen to be overridden. To facilitate this the PE object could
specify whether per job overriding of the allocation rule is
forbidden/allowed/required. All of these three modes seem to be useful.


To facilitate (1) the -pe_alloc switch should have only *one* option
argument

   "-pe_alloc <allocation_rule>"

and would never be accepted without a

   "-pe <pe-name> <slot-range>"

request.

To facilitate (2) a new attribute 'override_allocation_rule' should be
added to the PE object. The values accepted for this attribute would
be "forbidden"/"allowed"/"required".


   ------- Additional comments from andreas Wed Apr 16 02:37:11 -0700 2003 -------
The draft sketched under "HOW TO FIX" is not sufficient.
It must be reworked to allow user defined allocation
rules be used in a consistent combination with other features
of the PE object.

These are
* administrator controlled PE start/stop procedure
* administrator controlled access to the PE
* administrator controlled maxmimum amount of slots for a certain
  group of parallel jobs
* administrators decision whether for those parallel jobs
  the tight/loose integration must be used (!)

To allow this the draft should be reworked to allow jobs be rejected
at submission time in case

(1) the job requests an allocation rule without also refering to an
administrator provided PE object. By doing so one ensures e.g. that it
is specified whehter tight/loose integration must be assumed for the job.

(2) the PE object refered by the users allocation rule request is not
foreseen to be overridden. To facilitate this the PE object could
specify whether per job overriding of the allocation rule is
forbidden/allowed/required. All of these three modes seem to be useful.


To facilitate (1) the -pe_alloc switch should have only *one* option
argument

   "-pe_alloc <allocation_rule>"

and would never be accepted without a

   "-pe <pe-name> <slot-range>"

request.

To facilitate (2) a new attribute 'override_allocation_rule' should be
added to the PE object. The values accepted for this attribute would
be "forbidden"/"allowed"/"required".


   ------- Additional comments from pollinger Fri Dec 9 08:39:15 -0700 2005 -------
Changed subcomponent