Re: Anyone interested in improving postgresql scaling?

[Kevin Way]

Robert Watson wrote:

On Tue, 10 Apr 2007, Kevin Way wrote:

Kris Kennaway wrote:

If so, then your task is the following:

Make SYSV semaphores less dumb about process wakeups. Currently whenever the semaphore state changes, all processes sleeping on the semaphore are woken, even if we only have released enough resources for one waiting process to claim. i.e. there is a thundering herd wakeup situation which destroys performance at high loads. Fixing this will involve replacing the wakeup() calls with appropriate amounts of wakeup_one().

Could this cause problem cause a situation where an 8-Core system was 50-75% slower than an otherwise equivalent 2-Core system?

I have a graph of my sysbench/pgsql results here:

  http://blog.insidesystems.net/files/sysctl-pgsq-amd64-wtf.png





Kevin Way Inside Systems, Inc.

As the graph shows, the 8-core system is about half the speed of the 2-core system at 2 simultaneous threads, and it decays down to approximately 1/4 the speed of the 2-core system as the # of threads hits 5. All other (non-pgsql, non-sysv) tests came back approximately as expected, but I'm left wondering if I did something wrong, or if 8 cpus are slower than 2, when it comes to Postgres on currently available FreeBSD. (Full detail of what I did is available at: http://blog.insidesystems.net/articles/2007/04/09/what-did-i-do-wrong )

Robert N M Watson
Computer Laboratory
University of Cambridge

Would you be able to re-run these tests trying a recent 7.x kernel? If so, make sure you build a non-debugging kernel, and try two variants: one with SCHED_4BSD, and one with SCHED_ULE. Another experiment that would be interesting is to try using device.hints to disable various numbers of CPUs on your 8-core system, and see how the performance graph changes as the number of enabled CPUs changes.

I built 7.0 as of 6 days ago, and ran the same test using 8-cores, ULE and 4BSD. The results are available at:

http://blog.insidesystems.net/articles/2007/04/11/postgresql-scaling-on-6-2-and-7-0



Kevin Way
Inside Systems, Inc.

Unfortunately, I can't run the additional tests, as the machines in question have now been deployed to production. I should have similar equipment available in a few weeks, and will try the other tests at that point.

smime.p7s
Description: S/MIME Cryptographic Signature