Apparently, Ben Giddings recently wrote: > Wesley J. Landaker wrote: >> What about some other extreme cases? Like, a class that plays a sound on >> a >> speaker? How do you unit test that? Or, likewise, any class that >> interfaces with the "world" "outside" your control? What if they are so >> complex that you can't easily or acurately model them? What about output >> that "needs" human interaction? > > I also regularly deal with cases that involve interfacing to external > systems. > > If I want to test something that grabs a web page, what happens if due > to network issues, my access times out? If what you're really > interested in is not related to the network, then you can either test to > see if you have the right URL and headers to fetch the web page, or fake > the fetch and see if your code handles the data properly, but what if > the web access is essential? Just to add some more fuel to the discussion: another real general place where this comes up a lot is graphics applications. From a real raytracer project I did: How do I know that my raytracer rendered a sphere correctly? Well, I look at the output image and say "hmmm... yeah, it looks like a 3d sphere..." -- there were a lot of things like that. I could unit test the guts, but often writing the tests themselves required first writing a little program to compute what the results should be... ;) Suffice to say, on that particular project, I ended up just unit testing some core pieces (where, arguably it was simple enough that I didn't really need the tests) and just doing ad-hoc testing on the larger pieces that did complex math or actual rendering (but, ironically, this is where I would have benefitted the most from test-first unit-tests, because it was so hard to get right!) > An example would be a bit of code that tries to use some data to figure > out what URL to use to get what it needs. The only way to test to see > if that URL is correct is to see if the web page it fetches contains the > data it expects. Because of the nature of the problem (the changing > web), you can't tell by looking at the URL if it is correct, you need to > see what it fetches. You could always retry if a network error occurs, > but what if the error is a 500 series HTTP error (i.e. your URL was > correct but the server is temporarily broken)? > > Are there general unit testing strategies for dealing with things > outside the system you have control over? I think it's pretty clear that there is set of "un-unit-testable" things. I wonder, though, if anyone has some brilliant ideas on how to mitigate this class of problems. So far what I have in my head is: 1) Don't bother unit testing (yikes!). 2) Do regression tests with "golden" results (i.e. compare against previous output -- this assumes no test-first, and that you at some point got it right and verified by hand--this, IMO, is a lot to assume) 3) Make use of mock-objects to simulate an external interface (only works if it's low-enough complexity for you to model) 4) ??? 5) PROFIT!! (Okay, maybe not these last two... ;) Wes