> Given such a tool, I wonder if there would be interest in a 
> performance test suite for a Rails installation.

We actually almost have such a tool. It's been "almost finished" by 
Florian Weber for quite a while. It serves as an extension to test/unit 
and can compile reports on reqs/sec and so on. This would go well with 
a sample app that did a bunch of standard things and we've had that in 
mind too.

Preferably, it would give you all the details, but also arrive at an 
aggregated number you could use for rough comparisons. So you could say 
that server A got a 24 req/sec rating while server B got a 110 req/sec 
rating. That would be very nice indeed.
--
David Heinemeier Hansson,
http://www.basecamphq.com/   -- Web-based Project Management
http://www.rubyonrails.org/  -- Web-application framework for Ruby
http://macromates.com/       -- TextMate: Code and markup editor (OS X)
http://www.loudthinking.com/ -- Broadcasting Brain