Maybe You Are Doing Too Much Test Driven Development

I came across this StackOverflow post from 2008 where someone wanted to know when a developer can do too much testing.  Simply brilliant summary by the father of test-driven development (TDD), Kent Beck:

I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it. I do tend to make sense of test errors, so I’m extra careful when I have logic with complicated conditionals. When coding on a team, I modify my strategy to carefully test code that we, collectively, tend to get wrong.

Different people will have different testing strategies based on this philosophy, but that seems reasonable to me given the immature state of understanding of how tests can best fit into the inner loop of coding. Ten or twenty years from now we’ll likely have a more universal theory of which tests to write, which tests not to write, and how to tell the difference. In the meantime, experimentation seems in order.

I always find interesting to read so many blog posts about how to do TDD, including unit tests, integration tests, functional tests, factories, mocking and on and on.  There are conferences dedicated to software quality and software craftsmanship where TDD is advocated as all-in and anything less is not doing the right thing.

Clients pay me to write code too and not tests.  I write tests just enough to know the critical pieces of my system are working….and continue to work.  Developers could easily spend double their time writing completely tested systems and I’m not sure the customer gets 2X the value or ROI.

I think experimentation is certainly in order and see what best fits your situation.  The biggest takeaway, step back and decide for yourself what’s the right amount of testing and don’t simply follow the pied pipers of TDD.   Lots of great feedback from the others in response to the original question in that thread and great food for thought.

UPDATE: If you want to hear another developer’s perspective on TDD and using it as-needed, Marco Arment discusses how he uses testing in his very successful business on the latest Build and Analyze.  Very timely to my post.

  • http://twitter.com/kenegozi Ken Egozi

    Amen. People do forget what they are being paid for (bringing value to their customers).

    Next topic – beautiful code :)

  • Joe Eames

    Although I don’t disagree with anything you’re saying, it’s what you’re not saying that bothers me. There are certainly cases where people are writing more tests than is advisable, but by far the vast problem across our industry is too little testing, not too much. And it’s way better to err on the side of too much rather than too little when it comes to tests.

    Also, you never mention the vast number of other reasons to do TDD besides “assuring correctness”. If you are only practicing TDD to create a test suite to show that your code is correct, then you are missing the point. TDD for the tests is like jogging for the view. It is one of the benefits, but not even the best one.

    See http://www.testdrivenjs.com/2012/10/08/tdd-is-there-really-any-debate-any-longer/ for a more comprehensive list of the benefits of TDD.

  • http://www.accidentaltechnologist.com Rob Bazinet

    Yes, you are correct…I was not pointing out those fine points. The main point I was trying to make was that so many people pursue the path to 100% TDD that they are not using their own minds to decide what the right level of testing is.

    I’ve seen projects not seen to fruition because the developer/team was too busy worry what the TDD gods might think if they didn’t TATFT. I just find it silly to allow yourself to be led down a path without really knowing why you’re taking the steps in the first place.

    You do make good points and I agree completely. Developers should take it all in and decide for themselves what works for themselves and their paying clients.