[GE users] To run same scripts on multiple hosts

arvindpetaru arvindpetaru at gmail.com
Fri Mar 5 17:12:28 GMT 2010


Thanks Reuti for the info.

reuti>> so, nothing is shared and accessible from all machines like /opt/ 
applications or the users' homes?
--No, this project is for performance testing and nothing is shared for each host due to NFS impacting performance results.

Reuti, I'm new to MPI/PVM programming.....can u provide me some examples or links for parallel programming
using sge.

Basically, I need to start and stop C-binaries at the same time .......but prior to that.....shell-scripts on each host can have slight delay thou......so, I beleive, I may have to implement MPI/PVM programming for the C-binary implementation part..... 

So, for running same scripts on each host....I can develop a new script to start locally-available script to run on each host which in-turn would trigger/call other scripts in the host.  

--Thanks Arvind!!

> Hi,
> 
> Am 04.03.2010 um 21:36 schrieb arvindpetaru:
> 
> > I'm new to SGE. I've installed SGE 6.2u5 on sparc solaris 10. I  
> > have an existing project which is similar to SGE design of Master/ 
> > Slave interactions. So want to integarte existing project into SGE  
> > environment.
> >
> > Brief overview of my existing project....
> >
> > At setup, user would identify the master and slave hosts and a  
> > installation script would install all the required K-shell scripts 
> > (around 30+) and couple of C-binaries on eah host. So, after the  
> > installation process, each host(including master-host) would have  
> > same directory-structure and files(scripts and binaries).
> 
> so, nothing is shared and accessible from all machines like /opt/ 
> applications or the users' homes?
> 
> 
> > After installation, each host would start its own locally available  
> > scripts in a particlar order to gather its own data and pass it on  
> > to Master-host. Then the master host would signal all hosts to run  
> > its binaries at the same time. So the C-binary is started and  
> > stopped at the same time on all hosts gathering required data-info  
> > and passed it on to master-host. Master-host would process this  
> > data from all hosts and completes it assigned task....
> >
> > Here, are my below questions:-
> >
> > 1> How do I ask SGE to start running same scripts on each host at  
> > the same time ?
> 
> This is no foreseen - especially when you mean "at the same" time in  
> a sense of a "real-time" system. SGE will grant you a set of nodes  
> (better: slots) for a parallel job and start one script/binary on the  
> master node of a parallel job, the rest is up to you. All kids must  
> be invoked on your own by any type of MPI, PVM, Linda or other type  
> of parallel library.
> 
> MPI has such support for starting tasks on the nodes as PVM does.  
> When you setup these for a Tight Integration, SGE will also have  
> control of the started kids by the used `qrsh -inherit ...` to start  
> something on other granted nodes of this job.
> 
> If you want to use `qrsh -inherit ...`directly on your own, this is  
> also possible. The small challenge here is, that there is no real  
> async qrsh. But you can check the Tight Integration for PVM, where an  
> async wrapper is used to start the daemons on all nodes:
> 
> http://gridengine.sunsource.net/howto/pvm-integration/pvm- 
> integration.html
> 
> 
> > 2> Can I use a newly-developed script to initiate to use locally  
> > available scripts in each host to run.? If so, how to do that ?
> 
> Sure you can do that.
> 
> 
> > 3> Few scripts internally call other scripts...Does SGE have  
> > control/track these called-scripts? If not, what happens if the  
> > called-script fails? Does it get notified to the master-host ?
> 
> In a Tight Integration - yes.
> 
> 
> > 4> How to pass data from each host to master-host. I read abt  
> > prolog/epilog.....but how mcuh data it can support......my data  
> > gathering on each host is huge.
> 
> The is no built-in transfer of data. The prolog/epilog scripts are  
> started, and you have to program it on your own.
> 
> 
> > 5> Also, the C-binary spawns mutiple processes on each host based  
> > on the data obtained on it. But, here the criteria is that....C- 
> > binary is started and stopped at te same time on each host machine.  
> > Can we do this in SGE env ?
> 
> No, must be implemented by the user as suggested above.
> 
> -- Reuti
> 
> 
> > I'd appreciate your clarifications on the above questions/concerns.
> >
> > Thanks in advance.
> >
> > ------------------------------------------------------
> > http://gridengine.sunsource.net/ds/viewMessage.do? 
> > dsForumId=38&dsMessageId=247112
> >
> > To unsubscribe from this discussion, e-mail: [users- 
> > unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=247211

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list