[GE users] Allocation of specific resources.

Reuti reuti at staff.uni-marburg.de
Mon May 28 11:04:52 BST 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Hi,

Am 28.05.2007 um 10:02 schrieb Lönroth Erik:

> Hello again!
>
> We are slowly beginning to build up our SGE environment here at  
> Scania and the help from this list has been greatwhen dealing with  
> our initial problems.
>
> At this point, we are trying to get an application "powerflow"  
> intergrated into SGE. The nature of powerflow is that the  
> application can be divided into 2 steps.
>
> 1. Discretization. This job is not parallell, requires a single  
> node with alot of RAM. We have such a node - lets call it  
> "TheDiscMachine".
>
> 2. Decomposition+Simulation. In this step, it is optimal to run a  
> "controlling process" on a node with 16G RAM and the parallell  
> "simulation" on a number of CPU:s - lets say 128. The total number  
> of CPU:s involved will be 128(simulating) + 1(decomposing).
>
> How would you solve this situation in a good way?
>
> The application can take use of a "machine file" for MPI understand  
> the format:
>
> "CP    nodename1"
> "4	nodename2"
> "4	nodename3"
> ...
>
> Where the "CP" indicates where to spawn the "Controlling Process",  
> and the subsequent rows a <N> number of CPU:s to be used on the  
> corresponding node. The difficulty I think might be to get this  
> "CP" node to be automatically assigned to a specific node matching  
> my requirements (lots of RAM).

for now it is not possible to request such a configuration in SGE,  
i.e. different resource request for master and slave. Although it's  
alread an RFE (I think there is a second one in addition in the system):

http://gridengine.sunsource.net/issues/show_bug.cgi?id=75

***

I would more think of splitting the job. Blocking many machines in  
the cluster while the serial step is working seems not the optimal  
solution. Maybe you can submit a serial job for the first part, and  
then a parallel one for the second with "-hold_jid <wc_job_list>".

***

Another option might be: how many jobs of this type can run at once  
in the system, as obviously the limit is given by the feature(s) of  
the master machine(s)? You can setup the following, to allow a  
request like:

qsub -pe "extra*" 4 -masterq "master*" test.sh

You will need:

- one master queue for each core in the available master machines,  
slots set to 1
- one PE for these master queue, i.e. one PE per master queue  
attached with $round_robin or $fill_up
- one slave queue, containing all PEs and remaining machine, slots  
count set to the number of cores per machine

Machines node01/02 should always be the master machines:

$ qstat -g t
job-ID  prior   name       user         state submit/start at      
queue                          master ja-task-ID
------------------------------------------------------------------------ 
------------------------------------------
    6738 0.50031 test.sh    reuti        r     05/28/2007 11:56:41  
master1 at node01                 MASTER
    6734 1.00000 test.sh    reuti        r     05/28/2007 11:49:29  
master1a at node01                MASTER
    6735 0.99969 test.sh    reuti        r     05/28/2007 11:49:29  
master2 at node02                 MASTER
    6736 0.99969 test.sh    reuti        r     05/28/2007 11:49:32  
master2a at node02                MASTER
    6736 0.99969 test.sh    reuti        r     05/28/2007 11:49:32  
slave at node03                   SLAVE
    6738 0.50031 test.sh    reuti        r     05/28/2007 11:56:41  
slave at node03                   SLAVE
    6736 0.99969 test.sh    reuti        r     05/28/2007 11:49:32  
slave at node04                   SLAVE
    6738 0.50031 test.sh    reuti        r     05/28/2007 11:56:41  
slave at node04                   SLAVE
    6736 0.99969 test.sh    reuti        r     05/28/2007 11:49:32  
slave at node05                   SLAVE
    6738 0.50031 test.sh    reuti        r     05/28/2007 11:56:41  
slave at node05                   SLAVE
    6734 1.00000 test.sh    reuti        r     05/28/2007 11:49:29  
slave at node06                   SLAVE
    6735 0.99969 test.sh    reuti        r     05/28/2007 11:49:29  
slave at node06                   SLAVE
    6734 1.00000 test.sh    reuti        r     05/28/2007 11:49:29  
slave at node07                   SLAVE
    6735 0.99969 test.sh    reuti        r     05/28/2007 11:49:29  
slave at node07                   SLAVE
    6734 1.00000 test.sh    reuti        r     05/28/2007 11:49:29  
slave at node08                   SLAVE
    6735 0.99969 test.sh    reuti        r     05/28/2007 11:49:29  
slave at node08                   SLAVE


HTH - Reuti
---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list