[GE users] One slot per host, or a way to consume all slots

FL lengyel at gmail.com
Sun Feb 25 02:49:51 GMT 2007


    [ The following text is in the "ISO-8859-1" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

On 2/24/07, Reuti <reuti at staff.uni-marburg.de> wrote:
>
> Florian:
>
> Am 23.02.2007 um 18:40 schrieb FL:
>
> > I see from the link that I need to say more about gaussian parallel
> > jobs.
>
> we are also using Gaussian03 :-)
>
> > This method will work when the user submits a single-threaded job.
>
> You mean you want the complete machine for the job because of memory
> constraints?



And because of processor constraints: we have single P4s

> However, I do have a parallel execution environment for gaussian
> > parallel
> > jobs, and $pe_slots is set to the number of workerjobs-1; I see
> > that parallel
>
> To avoid misunderstandings: with $pe_slots I meant the setting in the
> definition of the PE. I.e. all slots must come from one and only one
> machine.



In my case, I may need slots to come from more than one machine,
but because this is a single processor P4 cluster, I don't want more than
one gaussian job or Lindaworker task running on any machine if there is
such a task running.

> gaussian jobs would need to set $pe_slots to (max slots per host)*
> > (workerjobs-1)
> > in order to consume the available slots in that case..
>
> This formula I don't get: why are you using (workerjobs-1)? If you
> refer to Lindaworkers, a parallel Gaussian job with a proper %
> lindaworker list and 4 cpu requested would also need "-pe linda 4".


The question of requesting the number of CPUs was determined by
observing that if your gaussian-Linda .com script has

%nprocl = n

then use

#$ -pe gauss nprocl-1

otherwise you get nprocl+1 tasks (a master + nprocl tasks).




To lay out your intention, you mean with a cluster of:
>
> node1: 4 cores, 16 GB RAM
> node2: 4 cores, 16 GB RAM
> node3: 2 cores, 8 GB RAM
> node4: 2 cores, 8 GB RAM
>
> to start just one Gaussian job with 4 processes, hence with one
> process running per node only?
>
> -- Reuti



If I had a cluster like that, then the users wouldn't be so concerned
about unrelated jobs running on the same node. But I only have single
(core) P4s with 1GB memory. (OK I have 6 core duos with 4 GB, but
this is another story--I would want the job allocation scheme to be
consistent: attempt to assign one Linda worker per core, and
exclude anything else).

Perhaps my understanding of the queing system is wrong: if I have
two queues A and B which can assign one slot each to host H (both
queues have host H in their host list) then the number of slots assigned
to H is 2 (the sum of the slots per host assigned to the queues) and
so I could get two jobs running on the same host H if they were assigned
from different queues.

Perhaps what I need to do is set the number of slots on each individual
host to 1, even though that will starve non-gaussian jobs that don't need
a whole P4...(or a whole core).

PS: Just to note, that there are some G03 tips on the list: http://
gridengine.sunsource.net/servlets/ReadMsg?listName=users&msgNo=14600



More information about the gridengine-users mailing list