Turbo transit tasks

[00:00 - 00:23] In this lesson, we're going to talk about turbo transit tasks, which I think is one of the least known parts of turbo and one of the most useful parts of turbo. I'm going to open my turbo.json, and I'm going to create a task that doesn't exist anywhere else in our project. It's only meant to be used internally in turbo.

[00:24 - 00:40] I have seen this task called either "topological," "topo" or "transit", so I like the transit word, I'm going to use that one. A transit task is a task that depends on transit.

[00:41 - 00:49] And you might be wondering what's the point of this. So this is not a real task that exists somewhere.

[00:50 - 01:09] But we have still defined it, and we have said okay, it depends on its dependencies running itself. This allows you to run tasks in parallel that would otherwise be running sequentially, for example in our package.json, we have the "test-types" task.

[01:10 - 01:36] I'm going to create a task called "test-types" that depends on "transit", and I'm going to replace it in here with "turbo test-types". "pnpm test-types", which is going to run in Turborepo and if we look at the output here, we can see that it started running test-types in parallel in all matching workspaces.

[01:37 - 02:09] Even though usually you would expect that the server express would depend on the logger, and Next.js would depend on UI, and similarly frontend-vite would also depend on the logger. What we did here is a way to circumvent sequential execution of dependencies where we don't care about outputs. When we're doing builds, we must first build the UI before we can build our frontend application.

[02:10 - 02:29] But when we're running something like linting, testing or type-checking, then we don't need to wait for the type-checking for the UI to finish to be able to run the type checking for our frontend application, we can run them in parallel. And this is what we did with this transit task.

[02:30 - 02:57] If we visualize this with "pnpm run test-types --graph" into our temporary folder "/tmp/test-types-graph.jpg", and I open this. What we're going to see is that all of these "test-types" tasks now go to the "transit" task. But the transit task doesn't have a definition anywhere in the repository.

[02:58 - 03:08] So there is nothing to execute. And turbo can immediately start start executing test-types, because in all the dependency graph there is nothing you need to execute ahead of it.

[03:09 - 03:20] But if there are changes. So you need to think about this in terms of if there is a file change, this would still propagate up and force this cache to invalidate.

[03:21 - 03:38] But because the transit task itself does not exist, it allows you to run everything in parallel. A lot of people will make the mistake of saying, of saying something like test-types depends on dependencies' "test-types" like this.

[03:39 - 03:57] And if we did that version and look at the graph. The graph is now that it needs to wait for the execution of type-checking for the other parts of the monorepo before it can run the leaves.

[03:58 - 04:10] So it needs to wait for UI to finish testing its types, before it can run Vite and Next.js. It needs to wait for the logger to finish testing its types before it can run the express-server type-checking.

[04:11 - 04:24] And we don't want that. So with that in mind, this is the way I suggest setting up your type-checking, linting and testing runs.

[04:25 - 04:36] And that's what I'm going to do. I'm going to then add "lint" depends on transient and also "test" depends on transit.

[04:38 - 05:04] And let me very quickly check that this works. So "pnpm turbo lint" is going to execute linting in parallel for the whole Monorepo and "pnpm turbo test" is going to execute the tests in parallel for the whole repo without waiting for the dependent tests to finish first.

[05:05 - 05:24] So with that in mind, we now have, these three and I usually like to create a common task in here that I like to call CI. And I usually create it with the following setup.

[05:25 - 05:43] I'll say "turbo run test-types lint test --concurrency=50%". So this is a very simple way to verify that our main checks run for the whole monorepo.

[05:44 - 06:02] This can be run with "pnpm run ci" which is going to execute all of these, and because we run them separately before they're all cached, so we get them executing in milliseconds. But this executes all the linting, testing and type-checking for the whole monorepo in one command.

[06:03 - 06:19] You should note that CI is a reserved word, so you must, when you want to run it, you must do it with "pnpm run ci". If you just do "pnpm ci", it's going to fail because CI is a reserved command in npm and pnpm has it reserved on its own end.

[06:20 - 07:01] And we're just saying that we want to run our script called "ci", and verify that everything in the monorepo behaves as expected. In terms of the additional setup that I did here, you might notice that we can run actually multiple tasks in turbo just by listing them all, and I'm passing an option of concurrency of 50%, because in my testing on most modern CPUs, this is the fastest execution you can get where you try to run your commands in half the processor cores that are available on your machine. At least in my testing, this performs the best.

[07:02 - 07:13] So concurrency 50% means run on 50% percent of the available CPU cores. And with that we can do "git add" everything.

[07:14 - 07:29] And our changes are that we now have the "ci" command and our "test-types" command uses turbo. We have defined "lint" and "test" commands as well. They all depend on our "transit" command to run in parallel.

[07:30 - 07:33] And we can commit this as "Parallel execution of CI scripts".