On Sun, Oct 07, 2007 at 12:38:21AM +0900, M. Edward (Ed) Borasky wrote:
> 
> 1. The site is very clear about what it is doing, how it is doing it, 
> why it is doing it and to a great extent how one can (not) and should 
> (not) interpret the results. However, anyone is free to misinterpret any 
> of that out of either ignorance or competitive marketing zeal.

I agree.  The site is useful -- just not in the immediately obvious way.
For that purpose, it's not useful (where "useful" includes "good at
providing strong technical evidence").  That's because the "immediately
obvious way" is in comparing languages to decide what should be used for
performance purposes between languages that are very close on the
execution performance scale, and micro-benchmarks are notoriously bad at
that, for a number of reasons.


> 
> 4. I suspect the main problem the "majority of the community" has with 
> the site is that it shows that the current (MRI) Ruby 1.8.6 
> implementation is slower than other languages commonly used to implement 
> web applications. And that is mostly competitive marketing zeal. Of 
> course the Ruby community wants Ruby to be faster than Perl, Python, PHP 
> or Java. And when it turns out that it isn't faster, there are two 
> things that can be done:

I don't think that's entirely the case (or even close-to-entirely).  I
don't just participate in the Ruby community -- I'm as much an on-again,
off-again member of a couple other language communities as I am of the
Ruby community here at ruby-talk (if not more so).  One in particular is
the Perl community, and just to make a point I'll compare the two on this
subject.

When people in each community look at comparative benchmarks for Ruby and
Perl, and when they see that Perl fairly consistently kicks the crap out
of Ruby for execution speed, I see that in *both* communities they tend
to follow that up with something like ". . . but micro-benchmarks are
pretty much useless.  Only real-world performance testing matters, and
usually even that is just a case of premature optimization."  In other
words, considering that even the usual "winner" in such a "contest"
dismisses those benchmarks as mostly useless in that respect, I don't
think the problem people in the "losing" camp have with the way people
tend to want to use benchmarks is primarily one of wanting to ignore what
shines unfavorable light in their direction.

I see the same things being said by Perlists and Rubyists: that
programmer productivity, flexibility, "real-world" performance,
maintainability, and other factors are more important.  In either camp,
people then tend to provide evidence that their chosen language is the
best for these factors in some way -- at least for the purposes of the
person talking.  As someone who likes and uses both Perl and Ruby, I find
that both "sides" are right, too.  They're just right for differet use
cases.

-- 
CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ]
Brian K. Reid: "In computer science, we stand on each other's feet."