[GE users] ncbi blast on SGE

reuti reuti at staff.uni-marburg.de
Wed May 26 11:12:33 BST 2010


Hi,

Am 26.05.2010 um 04:41 schrieb axinsoft:

> hmmm alright, masters, i give up the wrong expectation on SGE/blast. very appreciated you sharing your knowledge and exp.
> 
> the reason we try SGE & cluster is to submit millions of sequence for blasting. it will take a few month if we use normal single cpu workstation. 
> 
> i think i should take the way to blastall with "-a $NSLOTS"(this para should be wonderful). so SGE only take my job to run on a --single-- exec node (with NSLOTS cpus/cores), i should do a small script to cut my million seq to a few thousands seq pieces and submit quite a few times to different nodes, am i going to right way now?
> 
> when we are talking about thousands of sequence blasting, i test against small swissprot db (only about 200MB), my memory is 1GB/3.5GB, it take up only like 300MB RAM. even with the 7GB NR db(i format it by 1G volume), only 600MB RAM is used. according blast manual, it should read the whole db into memory if there is enough RAM. something wrong? so in my scenario, the CPUs seems to be the only bottleneck? 
> 
> so what i need to do, find as many CPU as possible, add them into my cluster, feed them the same number of jobs. correct?

as I wrote: you have to setup a PE (parallel environment) and attach this to one of your queues; it's also necessary to adjust the number of slots for all of the machines, depending on the number of installed cores.

a) `qconf -ap smp` you can leave all settings, just edit the numbner of slots to match the number of all cores in the complete cluster (`man sge_pe` can explain the fields).

b) attach this PE to a queue of your choice and set the number of slots in this queue equal to the number of installed cores. When all are the same, you just need to adjust the single number which is in the line "slots". If it's different from machine to machine, you can also use hostgroups to shorten the setup.

Then request this PE with the desired number of slots (obviously the maximum count you have installed on one single machine) and you are ready to go.

-- Reuti


> ------------------------------------------------------
> http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258554
> 
> To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

------------------------------------------------------
http://gridengine.sunsource.net/ds/viewMessage.do?dsForumId=38&dsMessageId=258624

To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].



More information about the gridengine-users mailing list