Unit tests are production code
I was recently talking with a coworker who said that they write their unit tests in Groovy because unit tests are not production code so the language does not matter. Of course, I think Groovy is a fine choice for producation applications, but I am ignoring that point for the purposes of this post. I could not disagree with my coworkers comments more. Unit tests are as important as your code itself. I do not think that comment needs more explanation than that, only that I find it interesting that many people still see unit tests as a second class citizen in a code base.
The other interesting thing is that the current goal is to get 80% unit tests passing on all projects (the projects that I work on have 100% all the time, otherwise I would lose my mind). This is a strategy I disagree with. Let’s say that over time your project has gotten to a state where you only have 50% of the unit tests passing. As far as I am concerned, this is equivalent to having 0% of the tests passing. Why? Because developers see tests in two colors: red and green. You will quickly find that the 50% will continue to down because no one is actually noticing that they are breaking tests as they are making code changes. On a few large projects I have worked on, I have seen where a single failing unit test quickly becomes 20.
Assuming that because of business goals and quantity of tests to fix, that just fixing all of the tests immediately is not an option. A much better strategy, in my opinion, would be to ignore the tests that are failing and say that all projects must have 100% unit tests passing. Then, make your goal to be to work your way down from 50% ignored tests to 0% ignored tests over a reasonable period of time.
Eric Peters:
I couldn’t agree more — Unit tests are production code; and 100% of unit tests should be passing at each commit. However, you left out another key issue: code coverage by the unit tests.
I’m working on a large 10 year old software package with a great deal of legacy technical debt. Our build process will fail if the coverage drops at all. So, even though we’re still only marginally above 50% coverage (due to legacy code), no new code gets into the code base without 100% coverage. It’s quite effective, especially with a fully automated build/test processes triggered by each svn commit.
Was there justification for the 80% line of thinking? That stance sounds equivalent to saying that 20% broken is okay.
May 16, 2009, 12:42 pmehauser:
EP – I’m not sure what the reasoning behind it was, I just saw it in a goals document. I am sure the thinking was that we really need to get our tests fixed, which is a good goal. I’m just advocating that there is really one valid state for tests – green.
May 20, 2009, 12:25 am