Node.js was created in reaction to slow web servers in Ruby and other dynamic languages at that time.
These servers were slow because they were only capable of handling a single request at a time. Any work that involved I/O (e.g. network or file system access) was "blocking". The program would not be able to perform any work while waiting on these blocking resources.
Node.js is able to handle many requests concurrently because it is non-blocking by default. Node.js can continue to perform work while waiting on slow resources.
The simplest, and most common form of asynchronous execution within Node.js is the callback. A callback is a way to specify that "after X happens, do Y". Typically, "X" will be some form of slow I/O (e.g. reading a file), and "Y" will be work that incorporates the result (e.g. processing data from that file).
To translate this back into words: "After the window is resized, print 'window has been resized!'"
Here's another example:
Similarly, this can be translated to "After 5 seconds, print 'hello from the past'" This can be confusing for most people the first time they encounter it. This is understandable because it requires thinking about multiple points in time at once.
If we were to run the above code, you would immediately see "hello from the present," and 10 years later, you would see "hello from the past."
setTimeout() is non-blocking, we don't need to wait 10 years to print "hello from the present" -- it happens immediately after.
Let's take a closer look at this non-blocking behavior. We'll set up both an interval and a timeout. Our interval will print the running time of our script in seconds, and our timeout will print "hello from the past" after 5.5 seconds (and then exit so that we don't count forever):
processis a globally available object in Node.js. We don't need to use
require()to access it. In addition to providing us the
process.exit()method, it's also useful for getting command-line arguments with
process.argvand environment variables with
process.env. We'll cover these and more in later chapters.
If we run this with
node 01-set-timeout.js we should expect to see something like this:
Our script dutifully counts each second, until our timeout function executes after 5.5 seconds, printing "hello from the past!" and exiting the script.
Let's compare this to what would happen if instead of using a non-blocking
setTimeout(), we use a blocking
We've created our own
setTimeoutSync() function that will block execution for the specified number of milliseconds. This will behave more similarly to other blocking languages. However, if we run it, we'll see a problem:
What happened to our counting?
In our previous example, Node.js was able to perform two sets of instructions concurrently. While we were waiting on the "hello from the past!" message, we were seeing the seconds get counted. However, in this example, Node.js is blocked and is never able to count the seconds.
Node.js is non-blocking by default, but as we can see, it's still possible to block. Node.js is single-threaded, so long running loops like the one in
setTimeoutSync() will prevent other work from being performed (e.g. our interval function to count the seconds). In fact, if we were to use
setTimeoutSync() in our API server in chapter 1, our server would not be able to respond to any browser requests while that function is active!
In this example, our long-running loop is intentional, but in the future we'll be careful not to unintentionally create blocking behavior like this. Node.js is powerful because of its ability to handle many requests concurrently, but it's unable to do that when blocked.
setTimeout() exist is so that we don't block the execution loop and freeze the UI. Our
setTimeoutSync() function would be equally problematic in a browser environment.
What we're really talking about here is having the ability to perform tasks on different timelines. We'll want to perform some tasks sequentially and others concurrently. Some tasks should be performed immediately, and others should be performed only after some criteria has been met in the future.
In the next sections we'll cover some different ways that Node.js allows us to do this using asynchronous execution. Callback functions like the one seen in
setTimeout() are the most common and straightforward, but we also have other techniques. These include promises, async/await, event emitters, and streams.
Node.js-style callbacks are very similar to how we would perform asynchronous execution in the browser and are just a slight variation on our
setTimeout() example above.
Interacting with the filesystem is extremely slow relative to interacting with system memory or the CPU. This slowness makes it conceptually similar to
While loading a small file may only take two milliseconds to complete, that's still a really long time -- enough to do over 10,000 math operations. Node.js provides us asynchronous methods to perform these tasks so that our applications can continue to perform operations while waiting on I/O and other slow tasks.
Here's what it looks like to read a file using the core
fsmodule has methods that allow us to interact with the filesystem. Most often we'll use this to read and write to files, get file information such as size and modified time, and see directory listings. In fact, we've already used it in the first chapter to send static files to the browser.
From this example we can see that
fs.readFile() expects two arguments,
setTimeout() also expects two arguments,
This difference in ordering highlights an important Node.js convention. In Node.js official APIs (and most third-party libraries) the callback is always the last argument.
The second thing to notice is the order of the arguments in the callback itself:
Here we provide an anonymous function as the callback to
fs.readFile(), and our anonymous function accepts two arguments:
fileData. This shows off another important Node.js convention: the error (or
null if no error occurred) is the first argument when executing a provided callback.
This convention signifies the importance of error handling in Node.js. The error is the first argument because we are expected to check and handle the error first before moving on.
In this example, that means first checking to see if the error exists and if so, printing it out with
console.error() and skipping the rest of our function by returning early. Only if
err is falsy, do we print the filename and file size:
Run this file with
node 03-read-file-callback.js and you should see output like:
To trigger our error handling, change the filename to something that doesn't exist and you'll see something like this:
If you were to comment out our line for error handling, you would see a different error:
TypeError: Cannot read property 'length' of undefined. Unlike the error above, this would would crash our script.
We now know how basic async operations work in Node.js. If instead of reading a file, we wanted to get a directory list, it would work similarly. We would call
fs.readdir(), and because of Node.js convention, we could guess that the first argument is the directory path and the last argument is a callback. Furthermore, we know the callback that we pass should expect
error as the first argument, and the directory listing as the second argument.
If you're wondering why
fs.readdir()are capitalized differently, it's because
readdiris a system call, and
fs.readdir()follows its C naming convention.
Let's now move beyond using single async methods in isolation. In a typical real-world app, we'll need to use multiple async calls together, where we use the output from one as the input for others.
Async in Series and Parallel#
In the previous section, we learned how to perform asynchronous actions in series by using a callback to wait until an asynchronous action has completed. In this section, we'll not only perform asynchronous actions in series, but we will also perform a group of actions in parallel.
Now that we know how to read files and directories. Let's combine these to so that we can first get a directory list, and then read each file on that list. In short, our program will: