On a recent episode (78) of This Agile Life, my fellow hosts talked about when to do Test-Driven Development (TDD). They all said that you should always do TDD — at least for anything that will go into production; there’s an exception for experimenting or “spiking”.
I wasn’t on that episode, but later commented on the topic. (Episode 83 — which wasn’t really all that great.) My take was slightly different. I said that you should do TDD only when the benefits outweigh the costs. Unfortunately, we usually greatly underestimate the benefits. And the costs often seem high at first, because it takes some time to get good at writing automated tests. Not to mention that both the costs and the benefits are usually hard to measure.
What is the cost of writing automated tests? I’ve asked this question before and recorded the answers (inasmuch as we have them) in a previous blog entry. TDD costs us about 10-30% in short-term productivity. The benefit that they found was a reduction in bugs by 30-90%, and a decrease in code complexity by about 30%.
But what about when the costs are higher than the normal 10 to 30 percent? One good example of this is when there’s no test framework for the situation you’re testing. This might be a new language or framework that you’re working with. More likely it’s a complex API that you have to mock out. So that could increase the cost of automated testing so as to outweigh the benefits — especially on a short project. I could imagine situations where the cost of writing the mocks could eat up more than the project itself.
Another case where we might consider skipping testing is when we’re more concerned about time to market than quality. This is almost always a mistake. Your code will almost always last longer than you expect. (Remember Y2K?) And if the code lasts longer than you expect, that means you’ll have to deal with bugs that whole time. But we have to work with the information we have at the time we make our decisions. And sometimes that might tell us that time to market is more important than anything.
One final case I can imagine is when a true expert is coding something that they’re very familiar with. I could picture someone like Uncle Bob writing code (in a language that he’s familiar with) without tests just as effectively as I could write code with tests.
But these situations should not be common; they’re edge cases. In almost all real-world cases, TDD is the right thing to do. Don’t forget, TDD is also a design discipline — it helps you design a better API. So keep doing TDD. But as with any practice, don’t do it blindly without considering why we’re doing it. Make sure you understand the costs, and whether the benefits outweigh the costs.