[GE users] Spooling benchmarks

jlb jlb at salilab.org
Thu Apr 15 00:50:14 BST 2010

On Fri, 9 Apr 2010 at 8:41pm, rayson wrote

> On 4/9/10, jlb <jlb at salilab.org> wrote:
>>  o I find the total throughput to be pretty low given the number of
>>    slots involved -- should it be higher?  Any hints as to where to look
>>    to improve things?
> How busy was the qmaster during the test?? Did it spend most of the
> time waiting for I/O or was it CPU bound??
> And did you play with any scheduler tuning parameters??
> "Scheduling-on-demand" should have an effect on the throughput if the
> qmaster machine can handle it:
> http://gridengine.sunsource.net/howto/tuning.html

It seems to me to be CPU bound.  All my initial testing was done with the 
following scheduler properties (which reflect the settings of my 
production cluster):

schedule_interval                 0:0:15
flush_submit_sec                  0
flush_finish_sec                  0

Doing a run of Test 2 (submitting and running simultaneously) with 32768 
jobs leads to the following graph of CPU usage:


(Note that, in the graph, 100% is 100% usage of a single core -- the 
server has 8 real cores and has hyperthreading on so that the OS sees 16).

I played around with all three parameters above and say only small 
throughput improvements.  The (roughly) best was simply:

schedule_interval                 0:0:15
flush_submit_sec                  1
flush_finish_sec                  1

This ran in 878 seconds (vs. 1076 seconds for the initial settings) and 
showed the following CPU usage:


So SGE is almost maxing out 2 cores.

Joshua Baker-LePain
QB3 Shared Cluster Sysadmin


To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].

More information about the gridengine-users mailing list