Benchmarking and speeding up pre-commit hooks

[00:00 - 00:45] In the previous lesson, we set up auto formatting on each git commit using pretty-quick. In that lesson, I asserted that pretty-quick is faster than running Prettier on the entire repository, but we never actually measured that. And since performance is crucial, especially for things like git hooks, where any regression to performance is going to be very noticeable to developers, we need to actually measure and characterize the performance of the tooling that we're using to do that. I'm first going to go into our package.json, and I'm going to add a "format" script that runs "prettier --write .", which means format all files in the repository and write the resulting output.

[00:46 - 00:58] I'm going to go back to the command line and then run this with "pnpm run format". We're going to see that it runs fairly quickly, but to measure it I'm going to add the "time" command in front of it.

[00:59 - 01:05] So "time pnpm run format". We can see that it takes just about 800 milliseconds.

[01:06 - 01:18] Now, while the "time" command is useful, it has some limitations. It will only run the command once, which might lead to some inaccuracies, and the way it displays information is not the best.

[01:19 - 01:27] Instead, we are going to reach for another tool called Hyperfine. I'm going to use "tldr" to show a short summary of how it's used.

[01:28 - 01:37] Hyperfine is a command line benchmarking tool. It runs a command multiple times to give a very accurate measurement of its performance.

[01:38 - 01:49] So let me run hyperfine and then in quotes the commands that we want to test, which is "pnpm run format". We're going to see it run the command multiple times.

[01:50 - 01:58] And it's going to give us a very accurate measurement of 780 milliseconds on average. Now let's compare this to pretty-quick.

[01:59 - 02:09] I'm going to run "hyperfine pretty-quick --staged". And then we're going to see that it completes in about 270 milliseconds, a significant improvement.

[02:10 - 02:22] It's already half a second faster. Even though this example Monorepo doesn't have a lot of files, you can imagine that over time, as more files are added, Prettier is only going to get slower.

[02:23 - 02:40] While pretty-quick, will maintain a more or less constant execution time even when you have tens or low hundreds of files changed in a commit, it's still going to be quite quick. However, there is still more that we can do to improve execution times here.

[02:41 - 03:07] And the main culprit right now is pnpm itself. The pnpm executable is quite complicated, so when we run a command line tool like Prettier or pretty-quick through pnpm, we're incurring a startup cost for pnpm itself. What we can do to avoid that is to use the npm binary scripts directly, rather than running them through pnpm.

[03:08 - 03:20] For every npm package that has a binary, we can use that binary directly. They're always created inside a folder in node_modules called ".bin".

[03:21 - 03:46] So let me quickly list that, and we can see that we already have binaries for lefthook, prettier, and pretty-quick in node_modules/.bin. We can open one of those binaries to understand how it works, and we're going to see that it's actually a shell script that takes care to provide the correct environment variables for loading dependencies of the given executable.

[03:47 - 04:15] And then all it does is run Node.js directly, executing the underlying JavaScript code, and passing all parameters down to it. To directly run one of these scripts, we can run "./node_modules/.bin/pretty-quick --staged", and we can see that this behaves identically to running it through pnpm.

[04:16 - 04:29] However, if we check its performance with hyperfine, we're going to find that it's significantly faster. It executes in just about 80 milliseconds, a 200 millisecond improvement, or close to four times faster.

[04:30 - 04:49] With that in mind, we should go to our lefthook.yaml file and we should change the execution command for auto formatting. Rather than using pnpm, we're going to do "./node_modules/.bin/pretty-quick" and then save the file.

[04:50 - 05:06] I'm going to add my changes to git, and then I'm going to commit them as 'Faster pre-commit formatting by using bin scripts'. With this change, our pre-commit hook now runs significantly faster and we maintain the exact same functionality.

[05:07 - 05:14] I highly recommend using this process of verifying performance for every new addition to your git hooks.