On Sun, Oct 07, 2007 at 03:02:15AM +0900, M. Edward (Ed) Borasky wrote: > Chad Perrin wrote: > > > >I don't think that's entirely the case (or even close-to-entirely). I > >don't just participate in the Ruby community -- I'm as much an on-again, > >off-again member of a couple other language communities as I am of the > >Ruby community here at ruby-talk (if not more so). One in particular is > >the Perl community, and just to make a point I'll compare the two on this > >subject. > > > >When people in each community look at comparative benchmarks for Ruby and > >Perl, and when they see that Perl fairly consistently kicks the crap out > >of Ruby for execution speed, I see that in *both* communities they tend > >to follow that up with something like ". . . but micro-benchmarks are > >pretty much useless. Only real-world performance testing matters, and > >usually even that is just a case of premature optimization." In other > >words, considering that even the usual "winner" in such a "contest" > >dismisses those benchmarks as mostly useless in that respect, I don't > >think the problem people in the "losing" camp have with the way people > >tend to want to use benchmarks is primarily one of wanting to ignore what > >shines unfavorable light in their direction. > > That's an easy way to dismiss the results, but you're assuming > > 1. That micro-benchmarks are pretty much useless. I don't agree. > Processors, compilers, interpreters, virtual machines and languages can > only be measured and optimized using micro-benchmarks. No, I'm not assuming that. I'm pointing out that objections to using those micro-benchmarks to choose a language to use for general purpose programming are common, and that I find them agreeable. Micro-benchmarks are *very* useful for determining problem areas in execution performance, if you happen to be on the core implementation maitenance team, but that's not what I was talking about when I said that as an overall performance measure for purposes of choosing a language they aren't so useful. In other words, the problem is that you're assuming that I'm assuming something that I am not, in fact, assuming. Put otherwise, I basically said that for the purpose to which most people seem tempted to put those micro-benchmarks when they drag them out while trolling, they're useless in any practical sense -- and you generalized from there to assume that I was saying they're never useful for anything. > > 2. That competition doesn't matter. Again, I don't agree. If it doesn't > matter, why do we do it? We long ago stopped racing horses because it > "improved the breed" and continue to race horses only because it is a > source of entertainment, and of tax revenue in some jurisdictions. But > we race computer chips and languages and algorithms because it *does* > improve the breed. Again, you're putting words in my mouth. I didn't say competition doesn't matter. What I suggested is that "my interpreter is faster than yours" competition based on micro-benchmarks is pointless for languages in the same general neighborhood of execution performance, and that usually other concerns are *far* more important within a given neighborhood of performance than execution speed. If you want faster execution speed, you should be in a completely different performance neighborhood, not quibbling over whether Ruby or Python is faster for an arbitrary set of micro-benchmarks, in other words. > > To sum up, as a practicing performance engineer, I can say that there > are ways to go from micro-benchmark results to "good enough but not > perfect" predictions of real-world performance. That's why we do it and > why it matters in a nutshell. Yes, we can -- but the margin for error is so great that, again, petty microsecond disputes between Perl, Python, and Ruby are just that: petty. I say this despite the fact that, overall, Perl kicks the tuckus of both these other close competitors -- and Perl is one of my favorite languages. I would have said roughly the same thing even before I started using Ruby. > > > >I see the same things being said by Perlists and Rubyists: that > >programmer productivity, flexibility, "real-world" performance, > >maintainability, and other factors are more important. In either camp, > >people then tend to provide evidence that their chosen language is the > >best for these factors in some way -- at least for the purposes of the > >person talking. As someone who likes and uses both Perl and Ruby, I find > >that both "sides" are right, too. They're just right for differet use > >cases. > > Well ... in the specific case of Perl vs. Ruby, I'm not by any means an > expert on the use cases that differentiate between the two. Ruby is by > design a *successor* to Perl, IIRC. So I would expect Ruby to provide a > better "user experience" than Perl for a lot of use cases. I don't know if it was intended that way, but I certainly don't view it that way. They are sufficiently different that they are simply *different* languages, not *successive* languages, in practice. Granted, I'd definitely be willing to use Ruby as Perl's successor within the specific niche of "object oriented programming", since Perl's OO syntax is so onerous to deal with (by comparison), but OOP is not the be-all and end-all of programming (the last fifteen years of Java industry domination notwithstanding). > > But I don't see how it's possible that language/implementation A can be > significantly faster than language/implementation B on a varied set of > micro-benchmarks and yet language/implementation B can be significantly > faster than language/implementation A on more than, say, *one*, > real-world application. Computers and mathematics just don't work that way. Who said anything about "significantly faster"? I'm just talking about the fact that Perl consistently kicking Ruby's tail in performance benchmarks doesn't mean a whole lot when choosing languages for projects, generally speaking, since if performance is enough of a concern to override the other factors involved you should be using something like C, Objective Caml, or at least compiled Lisp instead. In other words, if you're trying to decide between Perl and Ruby based on performance micro-benchmarks, you might as well be choosing between two paint shops for your car's new coat of paint based on which one has a better record for using paint that provides a finish with a lower drag coefficient. Obviously, it's such a miniscule difference in the grand scheme of things that you're better of deciding based on whether you like the color, whether a given shop's paint jobs tend to last longer, how much it'll cost you at one shop vs. the other, and so on. -- CCD CopyWrite Chad Perrin [ http://ccd.apotheon.org ] They always say that when life gives you lemons you should make lemonade. I always wonder -- isn't the lemonade going to suck if life doesn't give you any sugar?