Unit Tests are your Salvation
Unit Tests are your Salvation
You are not smart enough to understand the totality of your change set
Picture a small CRUD application with some basic business logic in the backend that interacts with a database. You have a known bug in one of the methods and need to push a fix as soon as possible for your end-users. After a few hours of testing and debugging, the change ends up as only one line. This gives you the confidence to push the change after local interactive testing passes your checklist. The error was removed and now users have the original behavior in place.
Shortly after, your inbox floods with user complaints that they are unable to update existing data in the application. You are confident that your previous change had nothing to do with the new error, so you spend hours debugging again, trying to figure out what the issue is. After struggling, yelling, and swearing, you finally decide to go back and revert your changes and confirm that the issue is now gone. The single line of code was in an unrelated area of the application, how did it suddenly break part of the business functionality?
If this story sounds familiar, it is due to almost every developer having this scenario happen to them in some part of their career. We try to pretend we are smart enough to understand the totality of our code, the totality of the given inputs, and the totality of the given outputs, and combine all this knowledge to predict if the application is satisfying the business use cases. But this is a flawed approach, for as much as we can try to predict the future, second and third-order effects are often hard to grasp and understand easily. The average person can memorize a simple set of numbers up to a count of 7 without mistakes. Now apply that idea to thousands of lines of code. This effort becomes impossible once a codebase becomes sufficiently complex, which is an extremely low barrier to reach. The simple truth is we do not have enough brainpower to flawlessly understand the changes to a program. We need not fight this limitation and instead learn to work around it. This is where rigorous unit testing of an application can spot flaws in an implementation far sooner than a human can.
Unit Test the Animals
At the most basic levels of software testing comes Unit Testing. A unit in this case means a small piece of code, testing means examining for expected behavior outcomes based upon inputs. For example, you have a simple abstract Animal Class and have a Dog class that inherits from Animal. As part of Animal, there is an abstract function called makeNoise(). For Dogs, they would bark, and for Cats, they would meow, and so on.
To test this, the first step is to ask yourself, “what is the goal of this test”? You want to make sure that when a Dog makes a noise, they bark. This is now your test case, and instead of performing this test yourself, it can now be put into a testing framework as explicit instructions to be run (ignore syntax specific things here):
[Test]
public void Dog_WhenMakesANoise_Barks() {
Dog dog = new Dog();
string noise = dog.makeNoise(); // returns "Bark"
Assert.IsEqual("Bark", noise);
}If you run this test, you will get a beautiful green arrow that your test passes since your Dog implementation barks. Of course, if you also change the Dog to meow instead, this test will fail.
In the future, if someone decides that is the correct behavior you can use the knowledge of the unit test and make an educated decision on the expected behavior of the system and discuss it with your team. Clearly, Dogs should not meow, but in other systems these lines are not so clear cut sometimes, and sometimes Product Managers are crazy. These unit tests serve as a reminder to express the expected behavior of the unit.
One thing to notice is how the unit test is human readable without any real programming knowledge. You can see each step and understand what it is trying to do at a high level. This declarative style communicates what behavior is being tested but not with specific implementation level ideas. These again are useful for conveying the expected behavior of a system to stakeholders, but also to other developers who will run into your tests at some point and want to rapidly understand the goals of a test. For more information on declarative vs imperative testing styles, read this LINK.
Now let us also add a similar test for the Cat class and ensure that they do Meow when they make a noise:
[Test]
public void Cat_WhenMakesANoise_Meows() {
Cat cat = new Cat();
string noise = cat.makeNoise(); // returns "Meow"
Assert.IsEqual("Meow", noise);
}Notice how there is a pattern with the previous Dog test. The main steps for testing are the same between tests, the only thing that varies is the object being tested has shifted. The basic setup of a test is a pattern from Test Driven Development that goes as follows (more info on Test Driven Development LINK):
Setup: Create our system under test (sut for short), in this case a Dog object.
Action/Execute: we perform the action we wish to test, calling the makeNoise() method.
Validate/Assert: Finally, we compared the expected outcome to the actual outcome and tell the testing framework what the result is.
Cleanup: Optional, depending on language, runtime features, and class design. Unit tests should not need to restore the state or the system to a certain way. If you do then there is a sign your code is designed poorly.
Depending on your language and test framework, we can actually supercharge this testing by cutting out all the repeated code for every implementation and create a single testing method that takes in varying Animals parameters to test:
[TestCase(new Dog(), "Bark")]
[TestCase(new Cat(), "Meow")] // Attribute specifying the test case
public void Animal_WhenMakesANoise_IsExpectedNoise(Animal sut, string expectedNoise) {
string noise = sut.makeNoise();
Assert.IsEqual(expectedNoise, noise);
}The syntax may not make exact sense, but the point is that now we have a singular method written that can test for every additional test case we can think of. If we add a Pig class, we just add a new Attribute to this test case and don’t need to repeat the entire method. This test is certainly basic, but realize that as development efforts and time goes on, your system could have thousands of basic tests for your application that would be impossible to manually review over in a reasonable amount of time. Your unit tests will certainly grow fast, so you must pick a testing framework that allows you to scale up fast, reduce repeated code, and is comfortable to work with.
What should we target when testing?
Now that we have written our first unit test, it is important to understand what exactly unit tests should cover in our code base. In general, unit tests should test every public facing API of a Class, meaning functions, properties, constructors, static methods, operators, etc.., as that is what consumers will be using when they interact with you. As for the inputs of tests, they should cover general, expected use cases, and also the critical edge cases. For illustrative purposes, think of your testing coverage of your class as a number line, where most of the general use cases can be covered by a healthy level of tests within the boundaries, but the boundaries are where the “interesting” (e.g. bugs) issues can happen with your API.
For example, if you are building a high level math library that does addition, subtraction, multiplication, and division, you can write unit tests for adding 1 + 1, 2 - 2, 3 / 3, etc. When testing addition, unit testing 1 + 1, 2 + 2, 3 + 3 makes sense. But does testing 11 + 11 make sense? Of course it does, but time is a limited resource, so maybe adding up every number in existence isn’t the most economical use of your time, or maybe that would take too long in your test runner. But something like an edge case that tests all potential bit overflows would be a useful edge case, such as an equation that resolves to 2^8, 2^16, 2^32, 2^64, etc. That doesn’t mean you can’t test the middle inputs of your number line, but focusing on important basic tests and edge cases would be prudent.
Additionally, most classes are not as simple as the Animal or Dogs ones we made above. You could have a complicated class with 5 dependencies needed just for it to be constructed and used. When it comes to unit testing, you want to test the class itself, not the dependencies! This can sometimes be a gray line, but ensure that any dependencies of your class just return either dummy data if possible, or provide a value back that our class under test consumes in its logical flow. We will cover this more in the mocking section below, but ensure that your tests do not try to test two classes at once. The dependencies should have their own dedicated unit tests also, independent of the current system under test. Unit tests should be kept as lean and focused as possible to gain the most value out of them.
One note is that if you ever find yourself needing to test private code inside your class, which for most languages is not easily testable due to access modifiers, it is a sign that your code is violating the single responsibility principle. Think about refactoring that logic into another small class and adding that as a dependency to your original class, and then unit test both classes. Notice the relationship between breaking code into smaller, testable pieces, and the modular symbiosis that results!
Mocking out Dependencies
With almost any class you write, there will be dependencies you rely on to enable certain functionality. These are typically things like databases, caches, libraries, and other classes or interfaces you’ve written and consumed. This makes unit testing much harder as now you have to supply these instances to just build the class you want to test. But you don’t want to unit test a class that needs a full database to work as that is tedious and inefficient. Additionally, if some of the behavior of your system under test delegates to a class you used, now you need to supply that whole instance, its dependencies, and so on. As you can see this is a massive problem to deal with, but fortunately there are class design decisions you can make early on to avoid this headache entirely.
What I am talking about is the D principle in SOLID - Dependency Inversion. Your classes should not depend on the concrete implementation of instances, but instead rely on the abstracted versions of them. This can commonly be implemented through using interfaces of the objects you wish to consume. Most libraries these days provide those interfaces for you, but if that is not the case, feel free to write your own thin wrapper on their API to enable this.
To provide an example, pretend we have a class that has a constructor that takes in a concrete database connection class, DbConnection. This DbConnection class as implemented requires you to instantiate a database connection to a live server. You must pass that into your class’s constructor before you even write a unit test. What if instead we programmed our classes to consume IDbConnection, allowing the actual concrete implementation of the database to not be known to us. This now gives us the flexibility to provide an alternative value to our classes as long as they satisfy the interface of IDbConnection. Now, you can hypothetically make your own concrete implementation of IDbConnection, called FakeDbConnection, but instead of making that real connection to a database, you no-op it! It does nothing instead! Suddenly your unit tests become much simpler to implement as now you don’t need a live database before attempting to run the tests for your code.
Unfortunately, we have gone too far in removing our concrete requirements with this approach. Having your dependencies do nothing can impact the overall logic of your class. We need to strike a middle ground where we remove the complicated concrete implementation but still want to provide useful values back to our system under tests to test the logic of them. We can update FakeDbConnection to return certain values when called upon, and customize it in the Setup phase of unit testing before we construct our sut. As you can see though, doing this for dozens of tests, and dozens of interfaces can be an effort in itself. Thankfully, this is where certain testing libraries such as Moq (LINK), which creates a mocked instance, or Jasmine (LINK) allow you to spy and stub out the returned value with no implementation overhead. This pattern, sometimes called a Test Double (LINK), should be repeated for all requirements of your class to create a simple and clean object for you to test properly. There is much more on this topic, but this is the most simple form of removing complicated dependencies from your code. Research the best option for your language take advantage of this testing pattern to make your unit tests easy to run while also making your code better.
Additional Benefits of Unit Testing
The biggest benefit of unit tests is the time saved from using an automated test system. You write once and run forever, ideally in your build pipeline on every merge to master and push on the remote branch. Every time you write a test that removes manual testing is time saved for as long as the product lives. This is how large companies like Facebook deploy software confidently, multiple times a day, to hundreds of millions of users.
Another benefit is that tests are a form of documentation for your product. Other members of your team can look at your tests to understand how new APIs are expected to behave, how to interact with them, and are an additional layer of documentation of the developer’s design. Besides the code itself, unit tests are often the last line of defense when reviewing old code to understand the developer’s intentions. This is especially true if you are working with Legacy code (for more info, read Michael Feather’s “Working Effectively with Legacy Code” LINK).
My favorite aspect of unit tests is the beauty when you make sweeping changes across your codebase and all the edge cases tests were tested, allowing you the comfort of addressing them while being able to wildly refactor code with confidence. Automated testing gives you peace of mind to make the right technical decisions within a codebase without losing the functionality consumers desire.
There is a concept called zettelkasten, sometimes meaning a “second brain” system of notes where knowledge is cataloged and linked together for recalling information. I like to think that Unit Tests are a form of this. They tell us how each class is meant to be used, documents using every public API, and as a bonus tells us if we are breaking something. Allow your unit tests to replace your manual tests and you will see your system become far more robust than you ever had it before.
That’s all for now, if you have questions please leave them in the comments and I will help you out the best I can!
Follow me on Twitter if you enjoyed this.
P..S. If you enjoyed this, feel free to send some tips (for ☕) at BowTiedCrocodile.eth