pgbench-tools is a set of program designed to turn pgbench into a useful benchmarking tool. The main historical problem with pgbench is that the results it generates are not particularly consistant from run to run. The approach taken here to cope with that is to run each test many times, with significantly more transactions than are normally used, then average the results out.
I haven't found anything better than pgbench for generating write workloads guaranteed to completely fill whatever I/O subsystem you have. If you run the no-branch variant of the test, which removes some locking issues, that turns it into a simple SELECT/INSERT/UPDATE set that will dirty the entirety of resources available in short order--shared buffer cache, operating system write cache, disk controller cache, you name it. This forces the system into a situation where every write is highly likely to turn into a blocking one, which one would be think is the ideal situation for the background writer to improve things--or get in the way.
The first problem is that you still have to size the database appropriately to keep seek behavior from dominating test results. I wrote something on that at http://www.westnet.com/~gsmith/content/postgresql/pgbench-scaling.htm and based on that did the tests on my server with a scale of 25, which puts the size of the database at around 367MB. Other major settings that impact pgbench performance here include:
shared_buffers | 32768 (=256MB) wal_sync_method | open_sync checkpoint_segments | 32 wal_buffers | 128 autovacuum | offThere's a full log of the non-default settings in the detail for each test, and the server hardware information is at http://www.westnet.com/~gsmith/content/postgresql/serverinfo.htm
Basically, the scale here makes a database that's big enough to not quite fit into either shared_buffers or the disk controller's cache, and what happens when you start pgbench with a decent number of clients is that you get somewhere around 1500 TPS for about 45 seconds when you start the test before all the caches fill. After that, results become a wreck because writes are quite regularly blocked, and the average over the test run hovers somewhere slightly below 1000 TPS. I sized the test runs to last as long as possible without hitting a checkpoint in most cases, but there's enough randomness that sometimes they sneak in anyway (particularly in the high client count tests).
The maximum latency numbers don't mean much in particular here because this is a pathological benchmark situation specifically designed to stress the background writer, and in most cases I get multi-second delays for many transactions without even getting to a checkpoint. What's nice about this particular test is that it easily exposes if you are writing inefficiently because that will drag down the TPS results significantly.
In order to get repeatable results, I run pgbench 5 times at 13 different client loads and average everything out; that's about 5 hours worth of data here. It's such a crude tool, that's the only way to get useful results from it--run it a lot, average, and eventually you'll converge on reality.
If you drill into the details, you can see that the tests slow down as the database is beaten on, even though I do some vacuuming between tests. To give a level playing ground, I reinitialize the entire database using pgbench -i before each test set run. This artifically inflates the first test run after that initialization, but again as long as I average enough results out that doesn't really impact the results significantly.