[GE users] Working of SGE_Hadoop Integration

templedf daniel.templeton at oracle.com
Tue Dec 7 13:08:23 GMT 2010


So, while the job is running, if you connect to the JobTracker page, 
it says that only one TaskTracker is running?

It sounds like the master task is failing to kick off the slaves.  Check 
for the output files from the job.  You should have <jobname>.o<jobid>, 
<jobname>.e<jobid>, <jobname>.po<jobid>, and <jobname>.pe<jobid>.  If 
the slaves are failing to start, there should be a clue in one of those 
files, probably the .po or .pe file.

Daniel

On 12/6/10 11:23 PM, adarsh wrote:
> I am using parallel environment of hadoop that is created after  ./setup.pl -i script.
>
> Here is my command :
>
> echo $HADOOP_HOME/bin/hadoop --config \$TMPDIR/conf jar $HADOOP_HOME/hadoop-*-examples.jar wordcount /user/hadoop/numb_complete.txt /user/hadoop/output6 | qsub -pe hadoop 3 -jsv /opt/hadoop_copy/jsv.sh -l hdfs_input=/user/hadoop/numb_complete.txt
>
> I want to now that is it depends on number of nodes ( qsub hadoop 3 ) whereas my nodes ( execution hosts ) are 8.
>
> However I also checked it with 1,8 no matter it runs on single hosts.
>
>
> Thanks for your Interest
>
> Adarsh
>
>
>
> Thanks for your interest.
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=302641
>
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=302768

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list