Paul Brannan wrote:
> On Tue, Nov 06, 2007 at 03:15:52AM +0900, Lloyd Hilaiel wrote:
>> Most of the time on this thing has been figuring out how quantify the
>> change, rather than making it.  It seems like having a robust, built
>> in "ruby  performance suite" could perhaps encourage future
>> exploration.
> 
> I agree.  Ruby 1.9 does have some performance tests in the benchmark/
> subdirectory, including a runner to compare performance to the
> already-installed version of the interpreter.  It's really only designed
> to eyeball speed improvement and isn't the robust performance suite that
> you're proposing, but it might be a good starting point.
> 
> Paul
> 
> 
> 

Well ... if it's just Ruby-implementation A vs. Ruby-implementation B, 
you can turn *any* test suite into a benchmark just by timing it! And 
every benchmark can also test for correct results, although most don't.

Maybe we could get people who have done "real-world" Ruby bottleneck 
chasing to give us their test scripts. I've got my own "collection", 
which includes all of the ones which come with Ruby 1.9, and my own 
Hilbert matrix inversion benchmark (which, by the way, *does* test for 
correctness, which is why it has to use Rational arithmetic.) :) And 
there's a "Pet Store" Rails benchmark I've got running, although it 
isn't integrated into the Ruby 1.9 driver test because it actually has 
two pieces (and I'm not sure Rails will run with 1.9 yet.)

Speaking of which, I've posted the PDF of my slides on RubyForge at

http://rubyforge.org/docman/view.php/977/2705/Slides.pdf

They're in the SVN repository too, but people told me they couldn't get 
Acrobat Reader and other fine products to open them that way, so I put 
them in the "docs" section of the project.