Austin Ziegler wrote:
> I can't think of a more useless "test" other than anything put out by
> the Alioth shootout.
>
> The numbers have limited interest because they're not necessarily
> using the *power* of the frameworks and merely measure the response to
> what is essentially a static return value. It would be more
> interesting to benchmark a simple guestbook in all of the things they
> did. Guestbooks are relatively easy to write and would be
> significantly more "test-worthy" than a "hello world" implementation.
My experience with web application benchmarks (nearly all
Windows/IIS/ASP/SQL Server and Linux/Apache 1.3/PostgreSQL/PHP) has
shown that the two most likely bottlenecks in a *real* web application
are the network bandwidth and the database. If the application generates
too much network traffic, or if its database design is poorly done,  it
doesn't matter all that much whether the underlying glue logic is in C,
Perl, PHP, Python, Java or Ruby, whether the OS is Windows or Linux or
some other, or what the web server itself is. The database, on the other
hand, does matter -- a *lot*. And that, my friends, explains why Larry
Ellison is such a rich man. :)

Having said that, the benchmark in the original poster's link, on the
other hand, measures the web server and a single module within it. I
think it's a perfectly valid benchmark for the specific web server/glue
language *component* of a web application, and I think there is a lesson
or two in the results for all of us:

1. C scales better than Perl, PHP, Python and Ruby, all other things
being equal. Make sure your C skills are up to date. :)

2. You probably want to hold off upgrading from PHP 4 to PHP 5, and you
want to performance test before you do.

3. Unless there is some compelling *business* reason to use one of the
other technologies, you probably want to avoid anything below PHP 4 in
this chart. Programmer time to implement the application is certainly a
compelling business reason. :)

4. The Ruby community needs to get Ruby's performance up where PHP 4 is
on benchmarks like this. It would be wonderful if it was better than
Perl and PHP, but a bare minimum is to be competitive with PHP 4.

On 4, I'm not sure a "virtual machine" is the answer, by the way.
"Virtual machines", or as I prefer to call them, "abstract machines",
were primarily intended for portability, not performance. C happens to
be a great abstract machine, and GCC happens to be a great way to
achieve portability and performance.


-- 
M. Edward (Ed) Borasky

http://linuxcapacityplanning.com