What should you do when testing is a pain?
Have you ever sat down to test your code and realized it’s going to be an utter pain? It may be too difficult, boring, or repetitive, it may just be too confusing… whatever the problem is you just can’t get yourself motivated to write those tests. What should you do? The easy answer: stop procrastinating and just follow the rules. Testing is good for you, like eating boiled brussels sprouts. But the easy answer is often the wrong answer. Boiled brussels sprouts are inedible; roasted brussels sprouts are delicious. Maybe there’s another way to achieve your goals.
If you’re unhappy it’s worth examining why you feel that way. The answer may help you solve the problem. Perhaps you just need to take a break, or have a snack, or even take a vacation. Other times it’s the nature of the task itself that is the problem. Let’s look at some reasons why testing can be painful and see what solutions they suggest.
“This is going to be soooooo boring.”
If writing the tests is boring that is likely because you’re writing many repetitive, similar tests. You might be trying to write tests to cover all the possible edge cases in an algorithm. Luckily there is a tool that is good at repetitive tasks: software. Testing tools that automatically generate test inputs can simplify your tests and find edge cases you wouldn’t have thought of yourself. Quickcheck deriviatives are available for many programming languages. Personally I’ve had great success using Hypothesis, a QuickCheck-inspired library for Python.
“I’m not sure what the code should do.”
This is a common problem when doing test-driven or test-first development, where tests are written before code. But if you’re not sure what the code should look like you shouldn’t be writing the tests yet. Instead you should code a test-less prototype to get a better sense of the API and structure of the code. The prototype doesn’t have to be correct or even work at all: its purpose is to help you see the shape of the API. Once you’ve got a better understanding of the API writing the tests will prove easier.
“I’ve found a bug but it’s too difficult to reproduce with a test.”
If you find a race condition in threaded code it’s often impossible to prove you’ve fixed the problem with a reproducable test. And one race condition suggests there may be more. The solution: stress testing, running your code under high load in hopes of catching problems. Stress testing won’t prove that your code is bug-free, but at least it will catch some problems.
“This is going to take far too much work to test.”
First, ask someone for help: perhaps a colleague will have an idea. Next, look for a tool that will help you with your particular testing needs. Testing a web application? Lots of tools for automating that. Testing a Unix terminal application? Some people have at least thought about it.
What to do if you’re still stuck? If the code is important enough, if getting it wrong will get someone killed or destroy your company, you should probably just pay the cost for testing. But usually that is not the case and other constraints apply: you need to ship your software and don’t have enough time to figure out or build a reliable test. Some options:
- Check if the code is covered some other way. An end-to-end test may exercise large amounts of glue code that is quite painful to test on its own. If the end-to-end test fails then debugging the cause may be painful, but at least you have some coverage.
- Manually test the change. This doesn’t guarantee it will continue to work in the future, but at least you know it works now.
- Give up. Sometimes testing your code is just too expensive given your project’s goals.
Paying attention to your pain and thinking about it can save you a whole lot of unnecessary work. What do you think? Have I missed any other cases where unpleasant testing can be avoided? Send me an email and let me know.