Demystifying unit testing & TDD (originally: "Unit testing is simple")

Published , last updated

Unit testing is a polarising topic in software. When we are new to software, we typically use println or echo to verify the result of the function we had just written. That is a manual, repetitive and error-prone process, and gets longer and bigger as your codebase expands.

Unit testing effectively takes all that manual work and boils it down to the key principle of 'verifying it works as expected, automatically'.

Unit tests serve as a documentation tool of the project, allows to quickly onboard new manpower onto the project and saves countless hours in debugging issues and subtle bugs.

Test-driven development (TDD) is a unit-testing practice where you write a failing test ("an expectation") and then make it pass, producing working code as an effect. A Microsoft Study on TDD, found that using TDD decreased defects by at least 40 percent.

How do you get started? Unit testing des not require a testing framework, because it is a principle and a methodology. However, the key thing is to show there is a failure. When in the command line, typically it is through exiting with error code 1 but there are other ways as well, such as console logging that we will use in this example.

Try unit testing right now! (Browser JavaScript Tutorial)

We will do this in your web browser's developer tools console. Skip to the next section if you don't have the time

We will use JavaScript in the browser because it is available for everybody and has an integrated assertion function :-). You will need to open the Developer Console, typically Ctrl+Shift+J on Windows and Cmd+Shift+J on macOS (for Firefox, use K instead of J)

Press the Copy button below, and then paste into the console (always check what you are pasting, before pressing Enter, for security reasons):

function increase() {}

Then run:

console.assert(increase(2) == 3, `expected 3, got ${increase(2)}`)

Here you would see an assertion error. While this doesn't prove anything in theory, in practice this is a very important step in TDD unit testing. The reason is that it shows that the test is failing, and because of that you are certain that it runs.

I have come across many situations where this step of failure was skipped, and the developers didn't realise that the unit tests were not running at all, ending up shipping code that they thought was properly tested!

Then run this to implement the function for a base/static case (important):

function increase(n) { return 3; }

And now verify again with the original assertion:

console.assert(increase(2) == 3, `expected 3, got ${increase(2)}`)

Now you would not see an assertion error - it has succeeded!

Now add another test case, since our implementation only works for one case:

console.assert(increase(3) == 4, `expected 4, got ${increase(3)}`)

Assertion error!

Now let's implement our function for the generic case.

function increase(n) { return n + 1; }

Then run the assertions again:

console.assert(increase(2) == 3, `expected 3, got ${increase(2)}`)
console.assert(increase(3) == 4, `expected 4, got ${increase(3)}`)

Now suppose it turns out there is a requirement that from 25 onwards, each increment is done by 5 instead of just by 1; and it always snaps to the next closest multiple of 5; such that increasing 25 gives 30, and increasing 26 also gives 30. We didn't want to give you such a simple problem, after all! Let's add some more test cases, so it now looks like this:

console.assert(increase(2) == 3, `expected 3, got ${increase(2)}`)
console.assert(increase(3) == 4, `expected 4, got ${increase(3)}`)
// case of 24 - we want to make sure it still goes to 25
console.assert(increase(24) == 25, `expected 25, got ${increase(24)}`)
// case of 25 -- goes to 30
console.assert(increase(25) == 30, `expected 30, got ${increase(25)}`)
// case of 29 -- goes to 30
console.assert(increase(29) == 30, `expected 30, got ${increase(29)}`)
// case of 30 -- goes to 35
console.assert(increase(30) == 35, `expected 35, got ${increase(30)}`)

Running these assertions, you should expect errors. Now fixing them demands a bit more complex logic:

function increase(n) {
    if (n < 25) {
        return n + 1;
    } else {
        return 5 * (1 + Math.floor(n / 5));
    }
}

Try to run them again. Now you have a suite of tests that serves as proof of what the function is supposed to do. Should you find a new optimisation to the code, to make it more efficient, you will not be scared of causing regressions, because now you have encoded your expectations.

However, we notice here that we are getting very repetitive. Let's make up a small framework:

function checkF(f, table) {
    const results = table.map(([input, expectation]) => {
        const result = f(input);
        return [input, expectation, result, result == expectation];
    });
    const successful = results.filter(([_, __, ___, success]) => (success));
    const failed = results
      .filter(([_, __, ___, success]) => (!success))
      .map(([input, expectation, result]) => (`for input ${input}, expected ${expectation}, got ${result}`));

    if ( failed.length ) {
        const arr =
            console.assert.apply(null, [false, 'Failed for inputs: ', ...failed]);
    } else {
        console.log(`succeeded with ${successful.length} tests`);
    }
}

Now we can call it with a broken function:

checkF((x) => (x + 1), [[2, 3], [3, 4], [24, 25], [25, 30], [30, 35]]);

And then with the proper function:

checkF(increase, [[2, 3], [3, 4], [24, 25], [25, 30], [30, 35]]);

This is very much the same as Golang's table-driven tests.

Why are unit tests useful?

  • They scale your testing capacity and catch regressions when you change code.
  • They act as documentation of how to use the code and what the code would output.
  • They enable you to change your code with confidence and enable to test new implementations of existing interfaces.

What do you test? These tests look too simple to me!

There are many depths of testing, and this is the very first one. For demonstration purposes, we had to keep it simple. It's when your application code scales where unit testing really pays off. I would say a good rule of thumb is >15 LOC.

I normally approach test cases so that the next programmer to look at the code (could be me 6 months later!) would be able to understand what code is supposed to do straight from the unit tests, rather than analysing what it actually does & reverse-engineering.

The right testing strategy is composed of different types of tests:

  1. Unit tests (lowest level tests, what we did today)
  2. Integration tests (tests of external systems)
  3. Smoke tests (sanity checks that can be automated or done manually)

There is no exact ratio between how many tests you should have in your software. In the case of Scala Algorithms, I have:

  1. 1323 algorithm unit tests to ensure the algorithms are correct, as this is critical
  2. 128 application unit tests to check general code
  3. 96 integration tests to check connectivity with Stripe and the database.
  4. 1 manual smoke test when making any significant changes, I will try to do a full check-out and registration flow, because this is very important.

Some tests may not be worth writing because of the inherent complexity of verifying the thing you want to check.

What are test frameworks for?

To make testing more organised. You get:

  • Organisation
  • Detailed error messages
  • Cool beautiful GUIs
  • Detailed test reports
  • Human readable language
  • In some cases, test frameworks can find the smallest reproducible case for you

My favourite testing frameworks, by language

Language Framework Reason
JavaScript, TypeScript Mocha + Jasmine Very straightforward to use
Java, Kotlin JUnit 5 Gold standard, well integrated with Maven
Scala ScalaTest Gold standard, highly flexible
Python PyTest Supports TDD with pytest-watch
C# MSTest Built in, works with Live Unit Testing
Golang testing Very straightforward to use, built-in

Comments (originally from dev.to)

Lennart commented:

Sadly, most posts/introductions about unit tests are usually way too simple with easy pure functions that just calculate some number. You don't have this kind of logic in real-world applications.

I have to deal with way more complicated scenarios daily involving object graphs that require initialization, external dependencies, drag and drop, framework objects that you can't mock and so on. I've yet to read a tutorial (or even book) that covers these cases.

How do I deal with legacy code that is not necessarily bad, but doesn't use DI and is not as easily testable as it ideally should be? How do I test public methods that don't return a value, but change something deeper down the call stack (internally)? I'd really like to read an article about how to tackle these problems!

William Narmontas commented:

Great questions! I actually have a whole lot to say on this but that'd have to be a whole new article.

For greenfield, functional programming is the answer. You don't need to test "changes deeper down the call stack" because there's no such thing as a change. Pass a value, get a value back.

We've written large complex applications with two flavours of code:

  • unit-tested functional code.
  • functional/acceptance-tested mutable/side-effecting code, which wires up the functional code.

Both flavours test-driven, not test-after.

You say that this legacy code is not necessarily bad but missing testability. Assuming this, I say the following:

  • Approach it by repeatedly extracting as much purity from mutable parts as you can. You'll find there's a lot of code that does not actually need to be mutable.
  • Turn the left-over mutable parts into immutable parts.
  • Only do this incrementally and not in a single step. Small commits, small pull requests.

The cost of fixing is still less than rewriting from scratch. See Joel Spolsky's article about "Things you should never do".

ferOrtega commented:

Not only simple but with functionality like Live Unit Testing (Visual Studio 2017) is also awesome.

Alan Campora commented:

This is a good starting point! I'm not an expert on this topic, but unit testing seems easy when you test "pure" functions. What happens when you have different results for the same input? Or functions that just call functions and you have to start using spies? Would be great if you go deeper in those topics!

William Narmontas commented:

Great question!

Exactly! If you want to make your life as easy as possible, you'll have as high a percentage of pure functions as possible. This requires explicit effort.

While you can achieve purity in any language, functional programming languages let you achieve it far more easily.

Personally most side-effecting code I write is also externally facing code, which makes them Integration Tests rather than Unit tests.

I very rarely use spies/mocks and the like. Brittle.

Kyle Johnson commented:

I have a feeling you've got quite a lot more to say about this area than you let on in this post. It'd be quite hard for anyone to gain anything this post , on the one hand they'd have to understand how assertions fit into the bigger picture of unit testing in medium to large scale apps whilst on the other hand they don't have anything but a few simple assertion statements to work with - which they probably already knew in their day to day language

William Narmontas commented:

Good comment, your assessment is accurate! I'm however not sure how to express it better though.

I have another article coming on Medium related to building things incrementally though, should be out today or tomorrow.

Stein B. Johansen commented:

For function, I agree, but is this unit-test or integration testing?

If you run up your application, and test how the functionality works from an external application, I would not call it "Unit-test" (or as it should have been called: Class test).

You can debate that your function/microservice is a "unit", but I do disagree, especially if it comes with a web-server which most microservices do.

Davide Zanotti commented:

unit testing is far more complex than writing an assertion statement... this post is pointless :/

William Narmontas commented:

What do you have in mind? I'd like to know to improve the post :)

I do use test frameworks extensively and find people have severe resistance to testing because of the learning curve. So this is a simple alternative for the newbie.

Davide Zanotti commented:

What you define as "unit test" in your post are actually mere assertions. They are helpful to ensure the sound state of your internal code but not for documenting and ensuring that your code behaves correctly as expected (as in a unit test). So, I'm not saying that assertions in code are useless, I'm saying that it's not unit testing! moreover an unit test has usually multiple assertions because the goal is to thest a method (the unit) under different scenarios!

Eddie Naff commented:

hunh. To me this is by far the hardest part of development. I'm not a specialist though. Different strokes for different folks I guess

William Narmontas commented:

Why is that the case for you? I'd like to find out so I can solve your problem and make testing better.

I rarely ever have to debug or println any more - just use a test.

Eddie Naff commented:

Well, for me it's the tooling. Getting everything to work together. The point of your article seems to be not using things like Mocha or what-have-you but I personally would be afraid to test without it. Or rather, I have no idea how I would use your method day-to-day.

edA-qa mort-ora-y commented:

This post is a good reminder that you can start off simple and grow into a framework as you need it. Unit testing is ultimately about the tests themselves, and this gives you the straightest path to writing them.

Ryan Palo commented:

That's so awesome! I didn't know about the Bash or Node ones. I've wanted to have some kind of testing for some of my Bash scripts. Thanks for sharing!