On Mon, Aug 24, 2009 at 8:47 AM, Marco
Mastrodonato<m.mastrodonato / gmail.com> wrote:
> Urabe Shyouhei wrote:
>> So the next thing you should do is to recompile by yourself to uniform
>> compilation environment among them.  ¨Âõô ®®>
> Yes i could do that but will be a different comparison. I don't think
> there are many people who compile themself from the sources, on windows

Another thing to consider when running the benchmark runs to not write
the program output to the console (e.g. ./program > /dev/null or
program.exe > NUL).  Presumably the intention is to compare the speed
of the implementations calculating the result and not test how fast
the desktop environment can display the result.  While it may not in
all cases make a significant difference in the results, it is best to
remove that variable from the test.

For example, in one case I saw a benchmark that claimed something to
the effect of "look BRANDX is not slow: this program in BRANDX is only
slightly slower than C!".  But, on closer examination it took the C
program less than 1 second to calculate the result and it took the
BRANDX program a few seconds to calculate the result; but, in both
cases, it took cmd.exe many seconds to display the result.

Using your Java program* (10 runs):
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.   Std-dev
  158.0   178.8   196.5   195.6   205.5   249.0  27.37882

and, sending out to to /dev/null:
   Min. 1st Qu.  Median    Mean 3rd Qu.    Max.   Std-dev
  106.0   108.2   109.0   109.0   110.0   112.0  1.699673

[*] Modified to send the time output to standard error