[GE users] execution nodes scalability

Brooks Davis brooks at aero.org
Thu May 18 18:55:33 BST 2006

On Thu, May 18, 2006 at 12:27:43PM -0400, Rayson Ho wrote:
> I read the paper a long time ago... but if I recall correctly, they
> only used the command line interface to add/remove nodes, so it should
> be quite GE version independent. The core part of their stuff is to do
> load balancing across different GE clusters.
> However, in your case, you will only need to add/remove nodes when the
> node comes up, so some simple scripts to call qconf may work nicely.

FWIW, adding exec hosts is trivial.  I use the following in my script
to bulk add nodes:

        qconf -aattr hostgroup hostlist $FQDN @allhosts
        qconf -as ${FQDN}
        qconf -ah ${FQDN}

        mkdir -p ${SGE_ROOT}/${SGE_CELL}/spool/${HOST}
        mkdir -p ${SGE_ROOT}/${SGE_CELL}/spool/${HOST}/active_jobs
        mkdir -p ${SGE_ROOT}/${SGE_CELL}/spool/${HOST}/jobs
        mkdir -p ${SGE_ROOT}/${SGE_CELL}/spool/${HOST}/job_scripts
        chown -R sgeadmin ${SGE_ROOT}/${SGE_CELL}/spool/${HOST}

Other than installing a startup script (that I don't use becuase the
FreeBSD port installed a better one), that's all inst_execd appears
to do.  I'd probably just keep the directories around (2^16 * 4 extra
directories isn't that big a deal, just avoid typing ls without -f
(don't sort) in ${SGE_ROOT}/${SGE_CELL}/spool.  Alternatly you could use
local spool dirs and avoid the issue entierly.

-- Brooks

To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net

More information about the gridengine-users mailing list