[GE users] Helpers and Performance

Chris Dagdigian dag at sonsorol.org
Thu Apr 3 13:21:50 BST 2008


    [ The following text is in the "WINDOWS-1252" character set. ]
    [ Your display is set for the "ISO-8859-10" character set.  ]
    [ Some special characters may be displayed incorrectly. ]

Hi Laurence,

The proper answer to this question is entirely application specific --  
it all depends on how your parallel application communicates. To a  
lesser extent the method of communication transport also matters along  
with the network topology (ethernet vs. myrinet vs. infiniband vs.  
quadrics etc.). Some of those methods have more "overhead" or put more  
load on the host system than others.

If this is an application that you wrote than there is nothing  
stopping you from doing your own scaling tests on your own system --  
why not try various ratios and see what works best in your  
environment? If you are trying to run the "vendor" application then  
I'd at least start with their recommendations first and then see what  
happens when you change things.

Also if you are new to integrating parallel applications into Grid  
Engine the best advice I can give you is to get your application  
working, tested and benchmarked without Grid Engine first. There are  
many things that can go wrong in a parallel environment when first  
trying to get an application working -- taking SGE out of the mix  
during initial tests is a great way to simplify the initial  
environment. The end result is faster troubleshooting and testing.

-Chris






On Apr 3, 2008, at 7:03 AM, Laurence Moughan wrote:
> "When dividing a job in to several parallel jobs, a master job is  
> created to handle the communication between the parallel helper  
> jobs. When you get too much overhead communication, at some point  
> this ?eats up? the performance gain."
>
> We have 20 cpus in the sge cluster, so why shouldn't i have 19  
> helpers to my 1 main job ?
>
> Thanks
>
> Laurence
>


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list