[GE users] Question regarding Parallel Environment

Daniel Templeton Dan.Templeton at Sun.COM
Fri Aug 24 21:03:43 BST 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Felix,

Kogan, Felix wrote:
> Hi, all,
>
> I wonder, is it possible somehow to change the node allocation for the
> parallel jobs on the fly? I.e., initial user requested 10 slots at the
> parallel job submission but then wants to add 3 more slots to the pool?
>   

Nope.  That was a feature we talked about a couple of years ago, but 
it's not one we ever decided to implement.

> More general question: 
>
> We have sets of production jobs that need to run on multiple nodes and
> are parallel in a sense. That is, they must start and stop together and
> if one of jobs can't be submitted, the rest cannot run. I'm thinking
> about using a sort of semi-tight PE integration (something like mpi or
> pvm integration methods provided with SGE installation): parallel job
> submission allocates set of slots and then custom program spawns
> necessary subprocesses to the allocated slots using "qrsh -inherit". 
>
> Is there any other, more suitable way that you could suggest to reach
> the same goal? Obvious thing is to create the dedicated queues for each
> such job set, but this causes inefficient resource usage, as many nodes
> remain unused. Better way would be to have a contiguous pool of
> production nodes from which all production sets can tap, but we need to
> ensure consistent allocation and lack of interference between the sets
> (i.e. if one set allocated slots on a node, another set can't use the
> same node). Of course, proper sizing of the pool would be necessary, but
> this is a solvable problem...
>   

Not entirely clear on what you're trying to accomplish, but it sounds 
like a custom PE fits your requirements.

Daniel

> Thanks,
>
> Felix
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
>
>   

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list