Rafe asks about blogger’s test-driven development experiences. I’ve been messing around with XP on-and-off for the last month or two. Yoz and I are doing some long-distance pair programming via TightVNC, Leo, and Yahoo Messenger Voice Chat, which has been slow-going but fun. I haven’t used XP or unit-testing for large projects, but I’ve been thinking a lot about my experiences.
Unit testing is definitely the easiest part of XP to grasp, but the XPers have a point when they say that much of the benefit of writing tests before you code comes from synergies from XP’s other practices. One of the reasons why most of us don’t write as many tests as we should is because of the limited returns. You spend a long time writing little fragments of code, which only come into their own if a bad bug trips them up in a predictable way. Anyone who has got caught in the trap of writing test code to catch bugs, only to find themselves fixing bugs in the test code, knows that tests can be a real drag on the flow of programming.
Unit testing before you code ups that return on investment. Firstly, while you’re writing the test code, you’re also mentally fleshing out the interface and algorithms of the future class itself. Many times now, I’ve found myself wrestling – and solving – some of the biggest algorithmic problems I have to face in writing a method while I’m writing the early test code. This has a couple of effects: psychologically, coding up the tests seems much less of a waste of time. It feels like a useful warm-up to the final work, rather than a finickety post-facto dredge over your previous work.
More practically, it means that the final product itself is far less composed of the lumber of a a dozen experiments. The effectiveness of this is particularly true in XP’s iterative design process. There’s a minimal amount of upfront design in XP, but there’s design *somewhere*. One of the hidden places design gets done is throughout the testing process. Now, that may horrify you, but it definitely means there are rewards to writing tests first that may not exist in a less agile development environment.
It has another knock-on effect, which, I think, leads to more productive and pleasant testing. When I’m writing unit tests these days, I feel a strong pressure not to “code to the test”.
The initial temptation when first writing pre-code tests is to write the minimum needed for your code to work. For instance, the code we’re writing at the moment is a simple WAP client. We have a test which just “presses the button” on the WAP page, to see whether we’re actually activating a CGI script. Putting aside whether this is really a unit test or a functional test (I admit to not truly understanding the difference), our first bash at this had something along the lines of:
Yoz and I discussed this for quite a while. I felt uncomfortable. It seemed to me that the test added code to the final version, but the wrong sort of code. It just adds to the main method code which fulfills the test’s requirement, and little else.
I was a bit surprised by my reaction to this, but realised that it’s partly what XP teaches you: because you’re outlining the skeleton of your later design in your tests, code like this feels like wasted effort. The test sketches the outline of code, but it’s not code that’s a step toward your final end product.
In the end, Yoz won this argument, and did so by pointing out, I think, one of the problems of unit testing the XP way. XP emerges, in large part, from Smalltalk best practices. One of the great features of Smalltalk is that you’re working in a completely OOP environment. Access and introspection just aren’t much of a problem. If there’s an object out there, you can talk to it.
No matter how OOPified our current programming world is, that’s just not the case, and that poses problems for unit testing with suites like JUnit. Take a look at the example above. One way that we could improve the design of the code, and write a test that fits in with the main execution flow is something like this:
Here we really are testing something, without adding additional code. We’re seeing if opening the form’s submit URL passes data to our cgi code. To do that we get hold of the object that represents the CGI codes state, and peer in it to see if it received the form data.
Unfortunately, this approach is impossible without quite a bit of wrapper coding: the CGI script in our world is in a completely different process space, and lives and dies without being easily introspected by our test code. We could write a persistent CgiObject communication interface, but you wouldn’t normally do so when writing CGI code without a unit test.
This seems a random example, but I’ve hit the same problem in other places as well. Testing WxPython GUIs using PyUnit is difficult, for instance, because somehow you’ve got to interpose your tests into your Python app’s event loop. I have funky code to do this now, but it required some heavy extra coding. Unless you’re really enthusiastic about writing your own test framework code, such test framework hacking has got to feel like wasted time.
It goes without saying that in Smalltalk you hit these problems far more rarely. You’re all sitting in the same event loop, and all instantiated objects and classes are accessible.
Even with these problems, though, I still write my tests before my code. Why? Well, part of the reason is I rather like Smalltalky OOP: having easily examined objects I think does lead to better code, and it’s nice to have that kind of architecture fall out of the test code you’re writing. One of the recurrent problems with CGI code, for instance, is that it’s really tempting to write it in a procedural style. Splitting it up into individual objects makes things a lot easy, and unit testing helps build that discipline.
The other reason is, as the phrase goes, I’ve become “Test Infected“. My tests have now caught so many bugs, small or large, that it feels rash to write code without writing the tests first.
My infection was unconnected to XP practices. The first program I wrote tests for was the code that handles my mail archiving. I really, really, really didn’t want this to fuck up, so I carefully pre-wrote all my tests. Since I started using the code (about 700 lines of Perl), I’ve only hit one bug. I found the problem almost instantly by kicking up the test suite again. It turned out that the syntax of the GNU ‘date’ command had changed subtly between versions. It’s the sort of bug that would have taken days to track down without the testsuites.
My own feeling is that, if you’re uncomfortable with JUnit, just write tests the way you feel comfortable with. After a while, when you’ve received some outright benefits from using tests, you’ll feel more encouraged to get to grips with a more alien suite. In the end, the *Unit classes are just a fancy way of saying: test this, then this, then this, then this. And that’s all you need. For small programs these days, I’m spending more time in Python’s doctest .