On Jun 12, 2004, at 12:28, Sean O'Dell wrote:

> ariĄ­trarĄ­y
>  adj.
> Determined by chance, whim, or impulse, and not by necessity, reason, 
> or
> principle.
>
> There is no fundamental reason tests should be ordered alphabetically. 
>  I've
> demonstrated that it's only too easy to order tests any way I want, so 
> I know
> alphabetic is not out of necessity.  So then, what reason or principle
> determines that tests should be run alphabetically?  I see none.  Is 
> there a
> reason and/or principal that says they must be run with that order?  
> Please
> tell me if there is, and I'll retract.

Early in test/unit's life, test order was arbitrary (well, OK, they 
were actually run in the order that Module#public_instance_methods 
returned them... but that was pretty arbitrary). In general, this was 
not a bad thing, because unit tests that are dependent on test order 
are a pretty serious code smell. However, PragDave pointed out that 
adding a bit more predictability would be a good thing, so I started 
sorting the methods alphabetically. This allows one to order test 
methods via naming 'hacks', but they feel like hacks. Some people don't 
like that, but I consider it to be a feature, because THEY ARE hacks.

So yes, I can agree that sorting alphabetically is a fairly arbitrary 
decision (although I can't think of another order I would prefer in its 
place). However, the choice to not make ordering an important part of 
test/unit's function is not arbitrary. The only other ordering I have 
really considered adding random order, with the seed being printed out 
each run so that results can be duplicated. That would be useful for 
removing inter-test dependencies, while most other orderings I can 
think of would encourage them.

As I've watched the uses of test/unit grow and change over time, I've 
also considered adding more ability to run tests in specific orders... 
however, this is because I see test/unit being used in various places 
at an acceptance (or customer) testing level, where those needs are 
much more legitimate. It's a challenge to introduce something like 
that, though, and still encourage good unit testing practices. Perhaps 
it's time for test/accept?

Anyhow, I'm glad that Sean has released this new library... it will 
only make test/unit better (I've already gleaned a few ideas from it). 
And when I get a chance to overhaul the test/unit documentation, part 
of the overhaul will be an attempt to communicate not just what 
test/unit does, but also what it doesn't do, and why.


Nathaniel
Terralien, Inc.

<:((><