Unit tests are the new documentation

4Dec/058

Unit tests are the new documentation

Refactoring Handles Unanticipated Changes: “”

I mildly disagree with Sam and Phil.

“But as Sam points out, what use is better code if it does the wrong thing?”

Yes, but what happens to your unit tests when you refactor your code to meet the new requirements? Now, you not only have to refactor your code, but you also have to refactor your unit tests. Also, writing unit tests, first, last, or during, doesn’t guarentee that your code will meet the requirements after the first, second, or N refactor.

My feeling is that unit tests are the new “documentation”. Unless you spend as much time babysitting your unit tests as you do your code, they become hopelessly outdated and are instead written after the fact. Which really just means you’ve increased the amount of code you have to write by two-fold. That doesn’t include all the build scripts, and setup of the automated testing software.

Plus, I’ve always wondered, what unit tests the unit tests? How can you ensure that your unit test is testing your code correctly?

Filed under: General Leave a comment

http://haacked.com/ Haacked

Well yes, you do have to change/remove your unit tests for the now obsolete feature to add the new feature.

But how sure are you that your new code isn’t going to cause an error in other dependent code? That’s the benefit of unit tests.

And as for unit tests as documentation, yes, they do serve as great documentation. But they are compilable documentation that are self-verifying.

When you change code, your documentation doesn’t tell you that its out of date. But your unit tests do by virtue they either won’t compile, or they do not pass.
Scott

“But your unit tests do by virtue they either won�t compile, or they do not pass.”

That’s a great point. Unit tests do tell you that they’re out of date where documentation won’t. But if it’s crunch time/death march time, through no fault of your own (I mean, hey you’re doing everything correctly. Unit testing, refactoring, proper design, yadda^3. But your stupid manager moved the end date up 4 weeks because the client/stakeholder thinks it shouldn’t be that hard to do what you are doing). I think that unit tests, much like documentation, will be ignored until after the launch. You’ll see all the red bars and feel guilty, but you’ll have to keep plugging away at the production code.

“And as for unit tests as documentation, yes, they do serve as great documentation. ”

I was speaking of them metaphorically, but that’s another good point. Both unit tests and documentation are supposed to tell you what the program is doing.
http://haacked.com/ Haacked

“I think that unit tests, much like documentation, will be ignored until after the launch.”

I agree that NEW unit tests will get ignored. That’s one key component of agile methodologies that you do what is necessary. I personally believe unit tests save you time, but in a really really tight crunch, when delivering something is more important than its quality, yes new unit tests can be ignored, but they incur a design debt (http://haacked.com/archive/2005/09/24/10336.aspx).

However, I would hope that you do not ignore existing unit tests that suddenly don’t pass. That would indicate that code you are changing is introducing bugs into another part of the system. No matter how crunch you are, if those bugs are serious enough, you have to address them. Of course you can choose to ship with known bugs that aren’t critical, in which case you should simply make a conscious decision to add the [Ignore] attribute to those unit tests.
Scott

“That would indicate that code you are changing is introducing bugs into another part of the system.”

It might not. It might just indicate that you’ve refactored the methods those tests are based on. Most developers will say, “Oh yeah, that’s broken because I changed the calculation in that method. I’ll fix that test later AFTER WE’VE SHIPPED”. You’ll see lots of bugs resolved in bug tracking systems with comments along those lines.

My general feeling about metholodolgies is that anything that requires the developer to do more work than is required to write the code to put the application into production WON’T get done except by the extremely disciplined. Those few and fine folk will define the methodology as required processes for putting the application into production.
http://haacked.com/ Haacked

Well it depends on the practice. For example, pretty much everywhere I’ve worked I’ve seen a 100% adoption rate of source control, though you can argue it isn’t absolutely necessary to get code into production for many of the projects with small teams.

However, every developer I know sees the value in it and it becomes second nature.

I think TDD is that way once a developer gets the hang of it. I don’t see it as a methodology but a sound coding practice. But it does require a developer seeing the benefits.

The times I see the biggest benefit is when my unit tests catch a regression introduced by someone else. For example, recently someone made a schema change that they figured only affected their code. In fact, it subtly affected code I had written and was caught by my unit tests. That potentially saved me countless hours of debugging.
Pingback: you've been HAACKED
Scott

re: SCM adoption. I’ve seen that too. But how often do you see SCM used properly? Most often I see HUGE check ins by developers I’m working with rather than the usual branch-develop-merge scenario. I can walk around to 3 dev machines and find 3 different versions of the code sitting on their hard drive. They think, “just one more line of code THEN I’ll commit my changes.”. Then a bug pops up and they think, “Well, I need to fix that before I check in.”. And so on.

See, you’re pointing out the benefits of TDD. Which I agree with, hence when I said I “mildly disagree”. The cynic in me knows that TDD is counter to the manner in which most developers learn to program. It takes a lot of discipline to stick to a regimen like TDD, or MSF, or IEEE, or any other methodology. We programmers aren’t known for our discipline.
http://homepages.inspire.net.nz/~gambit/ Riki

‘But how often do you see SCM used properly?’

I think the trick to that is having bug tracking and using that. If you don’t – then sure, developers just treat SCM as a glorified merge tool. If, like at my current work, you have active use of a bug track tool, thats intergrated with SCM, you have to checkin the code against a bug. and BigAssMerges can be avoided.

Sadly intergrated bug tracking doesn’t seem prevalent in small shops

Riki
Scott

Agreed Riki, I’ve used Perforce with Bugzilla in the past and the combination has worked out great. The lack of any informal standard for bug tracking/SCM integration means fewer and fewer teams are using both effectively IMO.

Maybe Vault, Subversion, or Sourcesafe.next will provide some leadership in this area? At least exposing simple REST services would be a step in the right direction.