[GE users] execution nodes scalability

Andreas Haas Andreas.Haas at Sun.COM
Thu May 18 14:39:05 BST 2006


On Thu, 18 May 2006, Krzysztof Wilk wrote:

> Hi all,
>
> I have a large pool of execution nodes (e.g. 65,536). Only a fraction of
> them (e.g. 256) are used at once. I need to use only one hostgroup.
>
> Would it be possible to add 65,536 create host entries and then add them
> to the same hostgroup?

I see no reason why it shouldn't work. Though the number of concurrently
running execd's is limited by qmaster's filedescriptor limit, but this
one can tune.

> Do you think single GE master will handle thousand of execution nodes?

Sure enough.

> Will they be completely ignored by scheduler daemon if they are not
> referred by any queue/pe/hostgroup/project/department?

Yes. If no queue instances exist for these hosts, no job can
be scheduled on them.

> Will the master daemon be polling all the configured nodes for their
> availability periodically?

There is no polling. Grid Engine execd's send their load reports
in load report interval (sge_conf(5)) and thus qmaster workload
depends on load report interval.

Regards,
Andreas

---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list