[GE users] To run same scripts on multiple hosts
arvindpetaru at gmail.com
Fri Mar 5 17:12:28 GMT 2010
Thanks Reuti for the info.
reuti>> so, nothing is shared and accessible from all machines like /opt/
applications or the users' homes?
--No, this project is for performance testing and nothing is shared for each host due to NFS impacting performance results.
Reuti, I'm new to MPI/PVM programming.....can u provide me some examples or links for parallel programming
Basically, I need to start and stop C-binaries at the same time .......but prior to that.....shell-scripts on each host can have slight delay thou......so, I beleive, I may have to implement MPI/PVM programming for the C-binary implementation part.....
So, for running same scripts on each host....I can develop a new script to start locally-available script to run on each host which in-turn would trigger/call other scripts in the host.
> Am 04.03.2010 um 21:36 schrieb arvindpetaru:
> > I'm new to SGE. I've installed SGE 6.2u5 on sparc solaris 10. I
> > have an existing project which is similar to SGE design of Master/
> > Slave interactions. So want to integarte existing project into SGE
> > environment.
> > Brief overview of my existing project....
> > At setup, user would identify the master and slave hosts and a
> > installation script would install all the required K-shell scripts
> > (around 30+) and couple of C-binaries on eah host. So, after the
> > installation process, each host(including master-host) would have
> > same directory-structure and files(scripts and binaries).
> so, nothing is shared and accessible from all machines like /opt/
> applications or the users' homes?
> > After installation, each host would start its own locally available
> > scripts in a particlar order to gather its own data and pass it on
> > to Master-host. Then the master host would signal all hosts to run
> > its binaries at the same time. So the C-binary is started and
> > stopped at the same time on all hosts gathering required data-info
> > and passed it on to master-host. Master-host would process this
> > data from all hosts and completes it assigned task....
> > Here, are my below questions:-
> > 1> How do I ask SGE to start running same scripts on each host at
> > the same time ?
> This is no foreseen - especially when you mean "at the same" time in
> a sense of a "real-time" system. SGE will grant you a set of nodes
> (better: slots) for a parallel job and start one script/binary on the
> master node of a parallel job, the rest is up to you. All kids must
> be invoked on your own by any type of MPI, PVM, Linda or other type
> of parallel library.
> MPI has such support for starting tasks on the nodes as PVM does.
> When you setup these for a Tight Integration, SGE will also have
> control of the started kids by the used `qrsh -inherit ...` to start
> something on other granted nodes of this job.
> If you want to use `qrsh -inherit ...`directly on your own, this is
> also possible. The small challenge here is, that there is no real
> async qrsh. But you can check the Tight Integration for PVM, where an
> async wrapper is used to start the daemons on all nodes:
> > 2> Can I use a newly-developed script to initiate to use locally
> > available scripts in each host to run.? If so, how to do that ?
> Sure you can do that.
> > 3> Few scripts internally call other scripts...Does SGE have
> > control/track these called-scripts? If not, what happens if the
> > called-script fails? Does it get notified to the master-host ?
> In a Tight Integration - yes.
> > 4> How to pass data from each host to master-host. I read abt
> > prolog/epilog.....but how mcuh data it can support......my data
> > gathering on each host is huge.
> The is no built-in transfer of data. The prolog/epilog scripts are
> started, and you have to program it on your own.
> > 5> Also, the C-binary spawns mutiple processes on each host based
> > on the data obtained on it. But, here the criteria is that....C-
> > binary is started and stopped at te same time on each host machine.
> > Can we do this in SGE env ?
> No, must be implemented by the user as suggested above.
> -- Reuti
> > I'd appreciate your clarifications on the above questions/concerns.
> > Thanks in advance.
> > ------------------------------------------------------
> > http://gridengine.sunsource.net/ds/viewMessage.do?
> > dsForumId=38&dsMessageId=247112
> > To unsubscribe from this discussion, e-mail: [users-
> > unsubscribe at gridengine.sunsource.net].
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users