[GE users] Illumina/Solexa pipeline on large SGE 6.2 systems?

Heywood, Todd heywood at cshl.edu
Fri Sep 5 21:15:02 BST 2008


We use that same wrapper script, as it is specified in the Solexa/Illumina
documentation. It is submitted via "qsub -R y -pe make <numslots> <script>"

We had some issues where the pipeline crashed due to communication issues
response between tasksnodes, but adding these ssh options to the config
helped a lot:

rsh_daemon                   /usr/sbin/sshd -i
rsh_command                  /usr/bin/ssh -o ConnectionAttempts=5 -o \
                             ConnectTimeout=60

Currently, the main problem is that the SGE qmaster gets overloaded in the
Gerald part of the pipeline due to many short tasks starting and ending
constantly. The read buffer (as per qping) fills up.

Using 6.1 here, and probably will wait for an update or two for 6.2 before
upgrading.

Todd



On 9/5/08 4:02 PM, "Chris Dagdigian" <dag at sonsorol.org> wrote:

> Thanks Sean!
> 
> Nice find, I found the 'distmake on SGE' page here:
> 
> http://distmake.sourceforge.net/pmwiki/pmwiki.php?n=Main.SGEIntegration
> 
> Are there any writeups around that summarize what benefits or what
> problems are avoided when going from qmake to distmake?
> 
> On our end we've been using a qmake wrapper script that looks like
> this for the Illumina pipeline:
> 
>> #!/bin/sh
>> 
>> #$ -cwd
>> #$ -v PATH
>> #
>> ## We use this with the Illumina pipeline to distribute
>> ## the make commands across the SGE grid rather than deal
>> ## with the standard non-distributed Unix make program.
>> ##
>> 
>> qmake -cwd -v PATH -inherit -- recursive
>> 
> 
> 
> ... no SGE issues so far with this but I suspect our instrument data
> and cluster sizes may be smaller than the others posting on this topic.
> 
> -Chris
> 
> 
> 
> On Sep 5, 2008, at 3:55 PM, Sean Davis wrote:
> 
>> 
>> I should have mentioned that when we have use SGE for the pipeline, we
>> have not been using qmake, generally, but distmake, which is SGE
>> aware.
>> 
>> Sean
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
> For additional commands, e-mail: users-help at gridengine.sunsource.net
> 


---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe at gridengine.sunsource.net
For additional commands, e-mail: users-help at gridengine.sunsource.net




More information about the gridengine-users mailing list