On 2/23/2010 12:20 PM, Yotta Meter wrote:
> I'm seeing some performance issues I can't seem to get around. Your 
> solution for creating a method was great though. I'd try the last one, 
> but I think I'm seeing a fundamental problem with Test::Unit.
>
> Using the following code as the base reference:
>
> #!/usr/bin/ruby
>
> require "test/unit"
>
> class TestImplArchFile < Test::Unit::TestCase
> (0..10).each do |i|
>  (0..10).each do |j|
>   (0..100).each do |k|
>    define_method("test_i#{i}__j#{j}_k#{k}") do
>     flunk
>    end
>   end
>  end
> end
> end
>
>
> When 'flunk' is set to flunk
> Finished in 5.586734 seconds.
> 12221 tests, 12221 assertions, 12221 failures, 0 errors
>
> When 'flunk' is replaced with true:
> Finished in 0.482389 seconds.
> 12221 tests, 12221 assertions, 12221 failures, 0 errors
>
> If I manually generate all 12k tests as individual tests in a new file, 
> with each one flunking:
> Finished in 5.631014 seconds.
> 12221 tests, 12221 assertions, 12221 failures, 0 errors
>
> The above would indicate that the problem is not with define_method, but
> rather the ability of Test::Unit to handle a large number of test cases.
>
> As a comparison, if I just print out results in a log file, I get the 
> awesome result:
>
> Just using a class, no Unit::Test, not creating method, just print 
> method name
> Finished in 0.102 seconds.
>
> I couldn't get define_method going in a class to compare, it's a private 
> method in Method, and I really don't understand the Method rdoc example 
> using send.
>
> Is there some sort of optimization in Test::Unit that I need to set? 
> Seems like something is really wrong, where is all this time going?
>   
Nothings wrong, exceptions are just slow.  This is actually true of most
programming languages.  'flunk' fails by generating an exception and
Test::Unit proceeds to rescue that exception.  For some more information
on just how slow exceptions are, check out:
http://www.simonecarletti.com/blog/2010/01/how-slow-are-ruby-exceptions/

This is why and exception should almost never be the expected outcome
of running any piece of code.  Likewise, failure should never be the
expected
outcome of 12221 tests.  If you have 12221 tests failing, you have much
bigger
problems than a 5 second run-time on your test suite.