Unit Tests and Unit Test Downsides to Defensive Programming in Python

For an introduction and dive into the theory around Defensive Programming, check out the first guide in this series.

One of the last ways to protect ourselves from future bugs is to not forget past bugs. Old bugs have a tendency to creep back into long-lived code bases. This is usually a result of someone not understanding why something is written in a particular way and removing it to 'clean it up.'

This presents another scenario where it's useful to have another form of executable documentation. So, when an unsuspecting developer changes some code or removes something, there's running code to alert them of their mistake.

Typically people encounter unit tests in the context of Test Driven Development. This is a great concept, but often in practice, it's a bit too optimistic and difficult to follow (read: the customer wants code now). However, that discussion is for another blog post.

What we’re going to discuss in this guide is how to use unit tests to protect you from future and past bugs. Think of this as somewhat reverses the TDD testing concept into something a bit more pragmatic with less up-front time costs.

I propose you write tests AFTER you fix a bug. See this discussion for more reasons why.

This serves a few purposes that might not be immediately obvious:

Surely you included a nice commit message indicating how you fixed the bug, but don't stop there. It's likely that your commit message and comments are lacking something like:

• How did you test the fix?
• What exact scenario caused the bug?

This is where a good unit test comes into play. You already know how to test the bug. (You did test it before committing, right?) So, codify the scenario you tested with and let everyone benefit from your hard work.

Unit tests are an excellent place to spell it all out and provide documentation for a bug fix. You can not only explain how and why you fixed it, but how you tested it. This information can be very valuable if the bug creeps up again.

Don't forget, a passing test can be used as a clue to where the bug is not located.

It's not uncommon for a particular bug to sneak back into a code base. This can happen because of changing requirements, re-factoring errors, or any number of situations. You can catch regressions by writing a unit test for a specific bug fix and remembering to run your tests. Think of a unit test as giving yourself a get out of jail free card on a bug you might inject again later.

The main downside of unit tests is that it's easy to forget to run them. This is just a byproduct of the tests not be located directly with the code. This downside can be mitigated by practicing something like Continuous Integration using something like Travis CI and automating your testing.

Another downside is that unit tests are typically only executed in their own testing environment, which is a simulated environment with no real users.

This style of development is tough to categorize and unfortunately, there aren't any solid rules to say when to use what. So, I encourage you to keep the guidelines in mind.

The guidelines will lead to a subtle change in mindset. The mindset change is important, not the tools and mechanisms themselves. Eventually you'll make some mistakes by overusing asserts or logging and start to form your own style. Also, the requirements for every project differ so it's important to learn all the tools and combine them in ways that make sense for your situation.