[GE users] Per job limit of PE slots

Hristo Iliev hristo at phys.uni-sofia.bg
Thu Jun 21 13:58:12 BST 2007

    [ The following text is in the "X-UNKNOWN" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some characters may be displayed incorrectly. ]

Hi everybody,

First I would like to applogize if I someone already asked the same
question and it got replied but my search through the mail list archive
was not as fruitful as I would like it to be :(

We have a 4 node Linux cluster with 8-core nodes that runs a mix of
serial, OpenMP and MPI jobs. 3 nodes are used for batch processing only
and one node is reserved for interactive jobs. Grid Engine 6.1 is used.
I would like to limit each parallel batch job to certain number of slots
depending on the value of h_rt requested:
- short test/benchmark jobs (up to 1 hour) that can eat up to 24 slots
- medium length jobs (up to 1 day) that can eat up to 16 slots each
- long running jobs (up to 1 week) that can eat up to 8 slots each
I have set up three cluster queues (with different h_rt limit) spanning
all 3 batch nodes and set 'slots=8' in exechosts definitions to prevent
oversubscription. The OpenMP PE uses $pe_slots allocation policy so it
is automagically limited to 8 slots but I have some very hard time
trying to convince SGE not to let long and medium MPI use more slots
than the policy defines. I can successfully limit user slots in each
queue with resource quotas but I cannot do it on a per job basis. What I
would like to achieve is to allow two (or even three) long jobs from the
same user running in parallel on 8 cores each but to deny one long job
taking 7 or more cores.

Is it possible to have such setup on a clustered system or am I making a
general design mistake following my Torque background?

Sincery yours,

Hristo Iliev

    [ Part 2, "This is a digitally signed message part" ]
    [ Application/PGP-SIGNATURE (Name: "signature.asc") 196 bytes. ]
    [ Unable to print this part. ]

More information about the gridengine-users mailing list