Chad Perrin wrote:
> No, I'm not assuming that.  I'm pointing out that objections to using
> those micro-benchmarks to choose a language to use for general purpose
> programming are common, and that I find them agreeable.  Micro-benchmarks
> are *very* useful for determining problem areas in execution performance,
> if you happen to be on the core implementation maitenance team, but
> that's not what I was talking about when I said that as an overall
> performance measure for purposes of choosing a language they aren't so
> useful.

Well ... OK ... you agree with me that they are useful for core 
implementation. And I *still* think they're useful when comparing 
performance of languages for general purpose programming, provided the 
benchmark suite is comprehensive and the statistical comparison 
methodology is correct.

I further think that the Alioth Shootout Suite is comprehensive enough 
to distinguish between the performance of those languages for which 
*most* of the benchmarks actually ran. The trick is the statistical 
methodology you use.

Ideally, of course, you'd have a benchmark suite that matches the 
application set. *That's* where people get into trouble with benchmark 
suites -- they assume that a given suite, for example, the SPEC integer 
suite, is representative of their application.

Now I could look at the benchmarks themselves and throw out those that 
were obviously orthogonal to web servers when comparing languages for 
implementing web servers. But I don't think I need to go that far -- 
it's easier to just throw out the statistical outliers and come up with 
a set of numbers that's a general characterization of the relative 
performance of the languages. That's why I asked for the raw data (which 
I received, by the way).

> What I suggested is that "my interpreter is faster than
> yours" competition based on micro-benchmarks is pointless for languages
> in the same general neighborhood of execution performance, and that
> usually other concerns are *far* more important within a given
> neighborhood of performance than execution speed.  If you want faster
> execution speed, you should be in a completely different performance
> neighborhood, not quibbling over whether Ruby or Python is faster for an
> arbitrary set of micro-benchmarks, in other words.

Of course. I'm still in the process of crunching the numbers, but what 
I've seen so far is that MRI Ruby is *not* in the general neighborhood 
of Perl, Python or PHP, but YARV *is* in that neighborhood. I do need to 
tighten up the calculations slightly and get rid of some bogus points, 
but I'm not expecting a major change in that.

As for things in the "fast lane", I started with gcc, and there are only 
a few things faster than gcc. The one that jumped out and bit me on the 
nose was a *functional* language called "clean". I'm just about to 
download it and see what makes it tick (or to stretch the analogy, how 
tightly it's wound). And the slow lane has such snails as "groovy" and 
"rebol" lumbering away.

> Yes, we can -- but the margin for error is so great that, again, petty
> microsecond disputes between Perl, Python, and Ruby are just that: petty.
> I say this despite the fact that, overall, Perl kicks the tuckus of both
> these other close competitors -- and Perl is one of my favorite
> languages.  I would have said roughly the same thing even before I
> started using Ruby.

Well ... let me finish with the numbers, but I think you'll be surprised 
at how close Perl and Python are to each other and how far away MRI is 
from the two of them. It's not a "petty microsecond dispute" for MRI.

> Who said anything about "significantly faster"?  I'm just talking about
> the fact that Perl consistently kicking Ruby's tail in performance
> benchmarks doesn't mean a whole lot when choosing languages for projects,
> generally speaking, since if performance is enough of a concern to
> override the other factors involved you should be using something like C,
> Objective Caml, or at least compiled Lisp instead.

Of course. But if the choice is between "dynamic languages with a 
sufficient supply of competent developers available", for example, Perl, 
Python, PHP and Ruby, and the assumption is that they are equally 
productive -- you're just trying to assess how much hardware you're 
going to have to throw at it if your customer growth takes off -- you're 
going to want the fastest one. Or, more to the point, you're probably 
going to dismiss the slowest one and move on to the other decision factors.