[GE users] ncbi blast on SGE

reuti reuti at staff.uni-marburg.de
Tue May 25 21:40:05 BST 2010


Hi,

Zitat von foisysdiploide <sylvain.foisy at diploide.net>:

> Hi,
>
> Blast and SGE: looks can be deceiving... Ok, here goes my experience on this
> topic:
>
> - Blast coming straight from the NCBI is not MPI aware but is multi-thread.
> So on a single node, you can specify blast to use more than one core. The
> flag for blast is -a <int>, where int is the number of cores you want to
> use.
>
> - If you want to do MPI with blast, your only recourse is to use (as
> prefiously suggested) mpiblast:
>
>  http://www.mpiblast.org/
>
> - Performance wise, blast is bound how much RAM you have and by the I/O from
> your storage for accessing the sequence databases. Even with
> parallelization, accessing the dbs becomes the bottleneck.
>
> Personnaly, I have often found that mpiblast is one layer of complexity to
> much for simple clusters with minimal support (my case). In addition, most
> blast runs are quite speedy by themselves and from experience, writing the
> report is always the the longest part and to the best of my knowledge, this
> is not parallized by mpiblast.
>
> From experience, pseudo-parallel of multi-sequence blast requests and using
> the -a flag for the single request (via a web page) is a very powerful
> combo. Both can be easily done with SGE, using the -b y "blastall -a 4
> blabla" :-)

thx for clarification. In this case a PE with allocation rule  
$pe_slots is necessary to avoid oversubscription of a node (often such  
a PE is called "smp"). If you put it in a job script, you can use:

blastall -a $NSLOTS

and you will automatically get inside the jobscript the number of  
granted slots for your PE request delivered to `blastall`.

-- Reuti


> My own .02 cents and best regards
>
> Sylvain
>
> On 22/05/10 13:42, "[NAME]" <[ADDRESS]> wrote:
>
>> hi masters:
>>
>> I just installed Rocks clusters 5.3(the lastest) with default SGE   
>> package on a
>> small 2 nodes cluster for testing purpose. Everything seems working. But the
>> ncbi blast is not working with performance double as expecting.
>>
>> example here a long time ago, seems blast is working with SGE very well
>> already.
>> http://developers.sun.com/solaris/articles/integrating_blast.html
>>
>> follow the instruction to create the job, it submit to one node only.
>>
>> i try qsub with array option (-t 1-2), job is dispatch to 2 nodes to utilize
>> all cpus. but the execution time do not even reduce a second, compare to
>> running job in one node. (the test run is 15 min with 1000 seq blastx)
>>
>> i try other option -pe orte 4, then qstat could see the job also spread to 2
>> nodes, but one of the host the cpu do not boost up at all. so the   
>> performance
>> is still same as before.
>>
>>
>> I google and seems all other people are happy and quiet. Do I need to config
>> or setup anything?
>>
>> I install Rocks 5.3 with default setting. only config i did was to   
>> add 'execd'
>> into the qmaster host to make it exec node also. (as instruction in Rocks
>> guide.)
>>
>> ------------------------------------------------------
>> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258
>> 237
>>
>> To unsubscribe from this discussion, e-mail:
>> [users-unsubscribe at gridengine.sunsource.net].
>
> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258462
>
> To unsubscribe from this discussion, e-mail:   
> [users-unsubscribe at gridengine.sunsource.net].
>

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258526

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list