On Jun 11, 2004, at 9:48 PM, Charles Comstock wrote:

> On Sat, 12 Jun 2004, Jim Weirich wrote:
>
>>
>> Nathaniel Talbott said:
>>> On Jun 11, 2004, at 13:50, Jim Menard wrote:
>>>> David A. Black writes:
>>>>> You're implying, I believe, that Test::Unit runs the test methods 
>>>>> in
>>>>> arbitrary order.  It doesn't; it runs them in alphabetical order.
>>>>
>>>> Is that guaranteed by the API or should I not rely on it?
>>>
>>> It is guaranteed. Even if other orderings are added at some point in
>>> the future, running in alphabetical order will always be supported.
>>
>> Even if it is guaranteed, I recommend not relying on it in the tests
>> themselves.  Each test should be very loosely coupled to its siblings 
>> and
>> the order should not matter.
>
> Yes but I can think of a very simple reason someone might want to 
> reorder there
> tests a bit.  Right now there is a bug in ruby 1.8.2 that causes one 
> of the unit
> tests to segfault the interpreter when running an extension test.  As 
> a result
> the test suite never completes, so I only know what works before it.  
> If this
> happened in a unit test that was of my own product there might exist a 
> case in
> which I wanted to double check all my others tests before the complete 
> failure
> case.  Particularly if I was writing more specific unit tests to try 
> to figure
> out what was causing the segfault.

In your own tests you can always put
flunk("To be fixed later")

or something like that in the beginning of the segfaulting test or 
rename the test so that it does not run at all (I usually put 
underscore in front of the test name). This does not justify 
introducing the test order at all. There might be other reasons, 
though. I simply haven't heard of any good one yet.

By the way, with Test::Unit you have ability to specify the exact test 
method or a test class on the command line, even using regexps. There's 
a bug in test specification though, however it is easily fixed. I did 
not submit usual report, sorry. With this feature, if I have a huge 
test suite, I can concentrate on a particular problem I am working on 
right now, and when it finally passes run the whole suite, happily 
leaning at the back of my chair.

>
> I might also just want the reports on related components to be in the 
> same area
> when they fail if I change something.
>
> Clearly you shouldn't set up tests to depend on ordering but for your 
> own
> understanding of the output I can think of lots of reasons you might 
> want your
> own ordering.
>
> Charlie
>
>
>

Sincerely,
Gennady Bystritsky