Once an app is running in production, it can be scary to make changes. How do we know that a new feature, fix, or refactor won't break existing functionality?

We can manually use our app the way we expect a user would to try to find bugs, but without maintaining a comprehensive checklist, it's unlikely we'd cover all possible failure points. Unfortunately, even if we did, this procedure is likely to be too time-consuming to be practical after every tweak.

If we were the only developer on a project, we might have a good sense of what's most likely to break when we make a change. This might narrow down the parts that we'd need to manually check, but this benefit is fleeting. Successful services can run in production for years, and as time passes, it's difficult to keep all those connections in our head.

Manual checks go completely off the rails once we're part of a team. Any developer who is unfamiliar with parts of the codebase or is new to the project will not know all of the ways to interact with the application or which behaviors are expected. Without some automated way to make sure that new code doesn't break things, all changes become risky and slow to implement.

If we're working on a team, even if we didn't worry about breaking existing functionality, we'd want to verify that new code works on its own. If we're responsible for reviewing a teammate's code before merging into the production branch, it can be time-consuming to figure out exactly the new feature does and how to check that works properly in a variety of situations. Good communication can help, but it still takes time.

As you might have guessed, the key to making all of this better is to incorporate tests into our workflow. By running parts of our app in a controlled way we're able to quickly see if new features work as expected and old functionality doesn't break.

Testing 101#

Let's start with an example: we want to create a function that tells us whether or not a number is prime.

We start with a placeholder module:

module.exports = function isPrime (num) {}

And we can start using it to see if it works:

const isPrime = require('./01-is-prime')

console.log(2, isPrime(2))

console.log(3, isPrime(3))

console.log(4, isPrime(4))

console.log(5, isPrime(5))

console.log(9, isPrime(9))

console.log(200, isPrime(200))

console.log(1489, isPrime(1489))

console.log(2999, isPrime(2999))

Of course, when we run our example, our new module won't be very useful:

$ node 01-is-prime-example.js

2 undefined

3 undefined

4 undefined

5 undefined

9 undefined

200 undefined

1489 undefined

2999 undefined

However, that's easy enough to fix. We'll first add some functionality to our new module:

module.exports = function isPrime (num) {

  for (let den = 2; den < num; den++) {

    if (num % den === 0) return false

  }

  return true

}

Then, we'll run our example again:

$ node 02-is-prime-example.js

2 true

3 true

4 false

5 true

9 false

200 false

1489 true

2999 true

This is much better. What's great about this is that we can allow another developer to work on this module, and we can easily check to make sure that they didn't break anything. This will be very useful for refactors. As long as the example output correctly identifies each prime, we don't have to worry about a regression being introduced.

For example, let's add we have teammate who wants to add an optimization. They inform us that our module will look for divisors all the way from 2 up to the number being tested. They tell us that this is inefficient and the complexity of our algorithm is O(n) when it should be O(log n). They seem to know what they're talking about so we let them make a change so that we don't need to check as many numbers as before:

module.exports = function isPrime (num) {

  for (let den = 2; den <= Math.log(num); den++) {

    if (num % den === 0) return false

  }

  return true

}

Now we can just run our example to make sure that no errors were introduced:

$ node 03-is-prime-example.js

2 true

3 true

4 true

5 true

9 true

200 false

1489 true

2999 true

That's not right... 4 and 9 are not prime numbers. Good thing we caught this issue before we deployed it to production!

This also highlights an improvement we can make to our process. We caught this because we know from experience that 4 and 9 are not prime numbers, and it was easy to see that this output had true when it should have been false. What about 1489 and 2999 though? Without checking our previous run, would we immediately know what the expected output should be?

Before we fix our module, let's improve things to make our lives easier. We're going to create a new file that will do the same thing as our example, but we're going to add some additional behavior to make it obvious when there are issues with our code. Instead of just printing out the return values of our method, we'll highlight any of them that are incorrect, and we'll print out what the result should be. Here's what that will look like:

$ node 04-is-prime-test.js

*  2 should be prime

*  3 should be prime

X  4 should not be prime

*  5 should be prime

X  9 should not be prime

*  200 should not be prime

*  1489 should be prime

*  2999 should be prime

8 Tests

6 Passed

2 Failed

X  4 should not be prime

      expected: false

      actual: true

X  9 should not be prime

      expected: false

      actual: true

What's nice about this output is that we can see how many cases are incorrect and what we need to do to fix them. Luckily, converting our example to a test is simple:

const assert = require("assert").strict;

const isPrime = require("./03-is-prime-log");

const results = [];

isEqual(isPrime(2), true, "2 should be prime");

isEqual(isPrime(3), true, "3 should be prime");

isEqual(isPrime(4), false, "4 should not be prime");

isEqual(isPrime(5), true, "5 should be prime");

isEqual(isPrime(9), false, "9 should not be prime");

isEqual(isPrime(200), false, "200 should not be prime");

isEqual(isPrime(1489), true, "1489 should be prime");

isEqual(isPrime(2999), true, "2999 should be prime");

finish();

function isEqual(actual, expected, msg) {

  try {

    assert.equal(actual, expected);

    results.push({ msg, expected, actual, error: false });

  } catch (error) {

    results.push({ msg, expected, actual, error });

  }

}

function finish() {

  const fails = results.filter(r => r.error);

  results.forEach(r => {

    const icon = r.error ? "\u{274C}" : "\u{2705}";

    console.log(`${icon}  ${r.msg}`);

  });

  console.log("\n");

  console.log(`${results.length} Tests`);

  console.log(`${results.length - fails.length} Passed`);

  console.log(`${fails.length} Failed`);

  console.log("\n");

  fails.forEach(f => {

    console.log(`\u{274C}  ${f.msg}`);

    console.log(f.error.message);

  });

  process.exitCode = fails.length ? 1 : 0;

}

The main difference between our example and our test is that we're storing the result of each isPrime() call along with the expected behavior, and after all the calls we print out the results. Additionally, if any of our results don't match what's expected, we change the exit code to 1 to indicate a failure. This is useful for when we want to this test to work in tandem with other processes. For example, we can make running git push conditional on our tests passing:

$ node 04-is-prime-test.js && git push

Because git push follows &:, it will only run if node 04-is-prime-test.js exits with a code of 0. A failing test will exit with a code of 1, preventing further execution. To easily see this in action, we can display a message only if tests pass:

$ node 04-is-prime-test.js && echo "  This can only be seen if the tests pass "

For more information about exit codes (also called "exit status") check out the Exit Status page on Wikipedia.

This is incredibly useful for deployments. We can make sure that deployment commands are executed only if tests pass.

Now that we have tests in place, let's fix our module. Clearly, our teammate was mistaken about using Math.log() to optimize our code. However, they were correct that we can reduce the search space for factors. The real answer is that we only need to search from 2 up to the square root of the number being tested.

Once we modify our module, we can run our tests again, and we can even check to see if we see our secret message:

$ node 04-is-prime-test-pass.js && echo "  This can only be seen if the tests pass "

*  2 should be prime

*  3 should be prime

*  4 should not be prime

*  5 should be prime

*  9 should not be prime

*  200 should not be prime

*  1489 should be prime

*  2999 should be prime

8 Tests

8 Passed

0 Failed

  This can only be seen if the tests pass 

Nice work, we've created the basics of a test framework! Now when we refactor or optimize, we can have the confidence that we aren't going to break anything.

Async Tests#

As we know, while Node.js is single-threaded, much of its impressive performance comes from its ability to handle work concurrently. It can do this by performing synchronous work while waiting on slow IO tasks. Unfortunately, Node.js can't do the reverse: if the process is completely occupied by a synchronous task, it can't start any asynchronous work.

Let's say that we wanted to put our machine to work by finding large prime numbers, we might do something like this:

const isPrime = require('./04-is-prime-sqrt')

module.exports = function (nStart, count) {

  let n = nStart

  const primes = []

  while (primes.length < count) {

    n++

    if (isPrime(n)) primes.push(n)

  }

  return primes

}

Unfortunately, this would be problematic to use when writing a production server. For example, let's turn our isPrime() and findPrimes() methods into an API:

const URL = require('url')

const http = require('http')

const isPrime = require('./04-is-prime-sqrt')

const findPrimes = require('./05-find-primes-sync')

const port = process.env.PORT || 1337

http.createServer(onRequest).listen(port)

console.log(`Prime Server listening on port ${port}`)

function onRequest (req, res) {

  const opts = URL.parse(req.url, true)

  const handlers = {

    '/is-prime': handleIsPrime,

    '/find-primes': handleFindPrimes

  }

  const handler = handlers[opts.pathname] || handleNotFound

  handler(req, res, opts)

}

function handleIsPrime (req, res, opts) {

  const n = parseFloat(opts.query.n)

  if (!isFinite(n)) return sendJSON(res, 400, { error: 'Bad Request' })

  sendJSON(res, 200, isPrime(n))

}

function handleFindPrimes (req, res, opts) {

  const nStart = parseFloat(opts.query.nStart)

  const count = parseFloat(opts.query.count)

  if (!isFinite(nStart) || !isFinite(count)) {

    return sendJSON(res, 400, { error: 'Bad Request' })

  }

  sendJSON(res, 200, findPrimes(nStart, count))

}

function handleNotFound (req, res, opts) {

  sendJSON(res, 404, { error: 'Not Found' })

}

function sendJSON (res, status, message) {

  res.writeHead(status, { 'Content-Type': 'application/json' })

  res.end(JSON.stringify(message))

}

If a client made a request to find many large primes, our server would be unable to respond to any other requests until they were found. We can test this ourselves.

We'll use curl to time our requests using the following command:

curl -s -o /dev/null -w  "%{time_starttransfer}\n" $URL_TO_TEST

We use the following flags:

• -s silences the progress output

• -o /dev/null will take the response data and write it to /dev/null which will delete/hide it

• -w "%{time_starttransfer}\n" controls the format of the output, and we specify that we're interested in how long it takes before we recieve the first byte

If we put all of this together while running our server, we can see that the /is-prime endpoint is pretty fast and on my machine curl clocks the response time at 5ms:

$ curl -s -o /dev/null -w  "%{time_starttransfer}\n" "http://localhost:1337/is-prime?n=130"

0.005

However, if we try to find 1,000 prime numbers greater than 10,000,000,000 we have to wait a bit longer:

$ curl -s -o /dev/null -w  "%{time_starttransfer}\n" "http://localhost:1337/find-primes?nStart=1e10&count=1e4"

10.340

Now, we'll do both requests. First, we'll start the slower /find-primes request, and then a few seconds later we'll start the fast /is-prime request:

Here we can clearly see the issue. The request that should have only taken 5ms, took over 6 seconds to complete -- over 1,300x slower than it needed to be!

Let's see how we can use testing to fix this problem.

The first step is creating some tests for our findPrimes() module. Since we want to use the testing framework we created earlier in the chapter, it's time to extract that into its own module:

const results = [];

module.exports = {

  isEqual,

  finish

};

function isEqual(actual, expected, msg) {

  const pass = actual === expected;

  results.push({ msg, expected, actual, pass });

}