ogilthorpe / davie.textdrive.com wrote:
> >
> > Matthew Desmarais wrote:
> >> Hi Isaac,
> >>
> >> I've paid attention to some of the discussions on the shootout here,
> >> but I'm not sure that I've ever had any of my basic questions
> >> answered.  Maybe you can answer them.
> >>
> >> The site lists as its goal, "to learn about programming languages,
> >> compare their performance in various (possibly meaningless) ways and,
> >> most importantly, have some fun!"
> >>
> >> What is it that we are learning about programming languages?
> >>
> >> What do we gain from comparing performance in "various (possibly
> >> meaningless) ways?
> >>
> >> I don't think that I've learned much from the shootout, but it may
> >> well be that I'm missing something.  I do that a lot. :)
> >
> > Interesting questions Matthew.
> >
> > imo What we learn depends on how actively we participate, and what we
> > already know or assume we know, and many other factors.
> >
> > So I should ask
> > - did you try to make any of the programs faster or more elegant?
> > - did you try to write programs in a new language?
> > - did you know there was a language Clean?
> > - did you understand that relative performance would vary so much from
> > tcp-request-reply to k-nucleotide to n-body and mandelbrot?
> > - did you notice any patterns: slow for io, fast for number crunching?
> > - did you wonder why the Lua program used ~100MB for nsieve?
> > etc
>
> I don't think that you answered either of my questions.  If you have and
> someone can tell me what the answers were, I'd appreciate it.

I was trying to answer "I don't think that I've learned much from the
shootout".


Let's try again - "What is it that we are learning about programming
languages?"

I don't know what you learn about programming languages from shootout,
the website states "Our goals are to learn about programming
languages..." - the goals of the folk administering the shootout are to
learn about programming languages...


Let's try again - "What do we gain from comparing performance in
"various (possibly meaningless) ways?"

Some perspective on how performance varies between programming language
implementations and tasks.


> I guess I'm not sure why the site is called "The Computer Language
> Shootout".  To me, a shootout feels like a thing that pits two entities
> against one another in a competition.  The result of a shootout is
> definitely a winner and a loser, depending on the definition of victory.

History - Doug Bagley's Shootout begat Aldo Calpini's Win32 Shootout
begat Brent Fulgham's Shootout.


> When you discuss the site, you talk about it as an educational tool.  I'm
> all for that kind of thing, and your response to me is reasonable and if I
> were looking for help it would be helpful.
>
> The problem is the use of the word benchmark.  By claiming to offer
> "benchmarks", the site is purporting to offer measured and standard
> methods of comparisons between programming languages.  The many
> disclaimers show that these numbers should be taken with a grain of salt,
> but the site's keywords, "benchmarking fast programming language benchmark
> performance benchmarks shootout program" show otherwise.
>
> And so I think that the site, or more specifically its design, is very
> disingenuous.  The site offers itself as a home for objective comparison
> of the performance of programming languages.  There is no such thing, and
> so the site should not claim that there is.

The site provides multiple comparison programs, which show various
different language implementations "winning".

The site shows different ways to "win" - by CPU time, by memory use, by
LOCs.

The site provides a synthetic overall score and invites you to
"manipulate the multipliers and weights to make your favourite language
the fastest programming language in the Shootout" for "a solution that
is simple, neat, and wrong".

Does the site proclaim A is faster than B, or subvert that simplistic
notion?