Robert Feldt <feldt / ce.chalmers.se> writes:

> phubuh / phubuh.org wrote:
>
>>
>>That said, I want to talk about testing general assertions
>>automatically.  By ``general assertions'', I mean assertions that
>>assert some property of arbitrary objects.  For example, say we wanted
>>to test this function:
>><snip lots of good stuff>
>>
> Nice! Glad to see some work in this area; I use random testing of
> properties often and it's a very powerful technique.

Indeed, it is!  I was kind of sceptical at the potential use of it at
first, but it turns out it's pretty easy to come up with general
assertions for lots of problems.  I think I'm going to be using it
lots.

> You might wanna check out the now a bit dated AutoTest stuff which
> seems similar to your RickCheck but takes a somewhat different
> approach using Generators that can generate different types of
> data. This seems a bit more flexible than your approach since you can
> have multiple Generators for objects of a certain type.

Ooh, that's interesting.  I think it's probably better to keep the
generation out of the classes themselves.  Or maybe the default
generator could be tacked onto the class, to make the common case
easier.

> OTOH, it's more verbose; would be interesting to merge our
> approaches

Definitely!

> and have it integrate nicely with Test::Unit (which I think yours
> might already be doing?).

Yup, it does.  It defines a subclass of TestCase called QuickCase,
which adds automatical testing of prop_* methods.  What I like best
about RickCheck is how easy it is to use for small stuff, and how it
blends nicely with Test::Unit:

| class MyTest < QuickCase           # most contrived example ever
|   def prop_foo(a)
|     yield [Integer]
|     assert_equal(a, (a + 1) - 1)
|   end
|
|   def test_bar
|     assert_equal(4, (4 * 2) / 2)
|   end
| end

But the use of yield seems weird now, having seen AutoTest.  In
retrospect, I can't believe I didn't think of just doing:

| def test_foo
|   assert_property(Integer) do |a|
|     assert_equal(a, (a + 1) - 1)
|   end
| end

I think that way is more natural, and it doesn't need as many changes
to Test::Unit -- test_foo is just a regular test.  And it's just one
line longer.  :-)

> You can check out the annoucement in [ruby-talk:10855] and the code
> is here http://www.ce.chalmers.se/~feldt/ruby/extensions/autotest/

Very nice.

> Related to this is also paper 2 in my thesis (available at
> http://www.ce.chalmers.se/~feldt/thesis/08_paper2_wise.ps) which
> evolves tests by combining test cells (basically Generators above)
> into more complex test behaviors. I am currently working on extending
> these ideas.

Wow!  Most of that is _way_ over my head, but it seems extraordinarily
cool.  Using biological ideas in computing always struck me as a
beautiful idea.  I'm glad we have guys like you on the Ruby boat. :-)

RickCheck used to do special treatment of `border cases'; for example,
Integer had this definition:

| class Integer
|   def border_cases
|     [1, 0, -1]
|   end
| end

But I removed it for some reason that I seem to have forgotten.  I'd
like to add it back in, but I'm not sure what's the best way to do it.
But it's certainly a Good Thing to test the border cases as soon as
possible; they're the most likely to screw up, after all.

Another thing I'd like to add is manual seeding of the random number
generator.  Every test run could tell you what it was seeded with, so
that you can easily reproduce it later.

Well, I've babbled enough.  Kudos!

--
/o\ /ッッッ\_
 o =      \_/
\o/ \___/ッッッ\_/