[GE users] How big is an extremly huge cluster?

ferstl ferstl at sun.com
Mon Dec 22 07:36:47 GMT 2008


what poll really improves vs select on Linux (Solaris does not have that 
issue) is the ability of the sge qmaster to handle huge frequency of 
messages. That in turn can have many reasons - and all of them in 
combination, potentially:

- Large number of hosts (several thousand is a ballpark figure),
   especially if the load report interval is set to comparatively low
   numbers (the default can be a low number in this context)
- Large amounts of finishing jobs per second on average (a result of
   short runtime jobs and comparatively large amounts of hosts being able
   to complete those jobs in parallel)
- Massive job submits
- A massive amount of status queries

So there is no simple answer to your question but, again, several 
thousand hosts can be considered a number where challenges certainly 
will occur and where tuning will be required.



Ben De Luca wrote:
> The beta release notes mention this,
> the use of the poll(2) system call instead of select(2) on Linux
> systems improves scalability of qmaster in extremely huge clusters
> I was wondering how big is extremely huge? 500? 1,000? 10,000?
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=93739
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list