[GE users] Spooling benchmarks
jlb at salilab.org
Sat Apr 10 00:07:37 BST 2010
A while ago I promised some benchmarks comparing classic vs. BDB spooling
over NFS on a decent sized cluster. Having been sidetracked by several
other things in the interim, I've only just finished them. I hope folks
still might find them useful.
Setup: SGE 6.2u4 courtesy binaries. Tests were done using both classic
spooling over NFSv3 and BDB spooling over NFSv4. All machines are
running up-to-date CentOS-5.
qmaster: HP Proliant DL160 G6, 2x Xeon E5520, 24GB RAM, single 250GB SATA
NFS server: HP Proliant DL580 G5, 4x Xeon X7350, 28GB RAM, 4x 15K RPM SAS
drives setup as a RAID10
cluster: 570 nodes of various speeds offering 3134 slots
Note that while the qmaster and NFS server were dedicated to this
benchmarking, the nodes themselves were still in use in our production
cluster. Every test was run 3 times and the time reported is the average
of those runs.
Submit a single array job with the specified number of tasks. Each task
simply runs /bin/true.
Submit the specified number of jobs, with each job simply running
/bin/true. Measure the amount of time it takes from the first submission
until the last job runs. This means that jobs are being run while more
are being submitted.
Submit the specified number of jobs, placing a hold on each job. Release
the hold, and measure the amount of time it takes for all the jobs to
A few observations:
o Load on the NFS server was generally low. It was never over 2 during
tests with classic spooling and generally around 0.5 during tests with
o Test 3 was hardest on the qmaster, with high loads and "failed gdi
response"s during the higher job count tests. This was true for both
classic and BDB spooling.
o I find the total throughput to be pretty low given the number of
slots involved -- should it be higher? Any hints as to where to look
to improve things?
I still have the test configuration setup (currently setup for BDB over
NFSv4), and I'm happy to try any suggestions folks have.
QB3 Shared Cluster Sysadmin
To unsubscribe from this discussion, e-mail: [users-unsubscribe at gridengine.sunsource.net].
More information about the gridengine-users