Course overview

Introduction

What is an Abstract Syntax Tree? Visualizing Code Like a Compiler

How to View Abstract Syntax Tree Code With AST Explorer

The Best JavaScript AST Tools - ESLint, Babel, Terser, and More

Environment setup

Understanding Abstract Syntax Trees (AST)

How to Generate a JavaScript AST With Babel Plugins

Traversing an AST With Babel Traverse

Add Type Safety and Prevent Runtime Errors With AST

Working with Abstract Syntax Trees

Practical code audits

How to Audit Your Code With AST Programming

How to Add a Component and Update an AST

When to Use Abstract Syntax Tree Tooling to Refactor at Scale

Code Audits

How to Mutate an AST and Automatically Replace Code Components

How to Create Codemods Using jscodeshift

Codemods

How to Ensure Codebase is Up to Date With Linting Rules

How to Use ESLint to Create AST Rules With Babel Traverse

Linting

node.js

In this course, we'll start with the fundamentals of abstract syntax trees (ASTs) and learn the basic mental models. This general AST knowledge can be translated to almost any tool that works with ASTs.

## Why this course?

Understanding and using ASTs unlocks the ability to make sweeping changes in a safe and reliable way in any size codebase.

## Course topics

Throughout this course, we'll have converted source code into ASTs, traversed, mutated, and generated ASTs. With these concepts we'll then explore several practical applications including things like code audits (static analysis), code transformations (codemods), and linting.

### Module 1

We'll learn the fundamentals of abstract syntax trees. 

- What is an AST?
- How to explore an AST
- Examples of JavaScript tools that work with ASTs

### Module 2

We'll learn how to work with ASTs.

- How to turn code into an AST
- How to programmatically navigate any AST 
- How to leverage TypeScript to prevent runtime errors

### Module 3

We'll learn how to statically analyze, or "audit" code to understand the state of the codebase using abstract syntax trees.

- An introduction to an example codebase and refactor
- Understanding the state of the current codebase
- When to use an AST-based tool versus doing something manually

### Module 4

We'll learn how to transform, or "codemod" code from one state to another using abstract syntax trees.

- How to make changes to an AST
- How to change ASTs with [jscodeshift](https://github.com/facebook/jscodeshift)
- How to test a code transform

### Module 5

We'll learn how to write rules, or "lint" code using abstract syntax trees.

- How to create rules for code
- How to create custom rules with [ESLint](https://eslint.org)
- How to test a rule

In this course, you'll learn the fundamentals of abstract syntax trees, what they are, how they work, and dive into several practical use cases of abstract syntax trees to maintain a JavaScript codebase.

Practical Abstract Syntax Trees

JavaScript tools that work with abstract syntax trees

How to parse, traverse, and generate abstract syntax trees

Practical skill set for maintaining large JavaScript codebases

The script from the previous module read files from the codebase, traversed the source code within those files to find the relevant code (nodes in an AST), and logged that data. With the data, we were able to define and build a component. Now that we have the component, it's time to go through and replace all the 

There are a few ways this could be approached:

All approaches are valid. The scope and complexity of the change can help determine which is the most efficient.

The button transformation needed here is complex. For example, the 

 string needs to be broken up into multiple props, but those props are only required if they're not the same as the default. Since the sample codebase only contains a handful of buttons, it would be realistic to manually replace them all. However, we're going to use AST-based tooling to understand how it can be applied to transform code.

Again, let's start by creating a new sibling directory to the Flash app, so initialize a new 

# Create a new directory for the transform script
mkdir transform

# Change into the new transform directory
cd transform

# Initialize a new package.json file
npm init -y

# Add the necessary dependencies
npm i @babel/parser @babel/traverse @babel/types @babel/generator glob ts-node typescript
npm i -D @types/babel__traverse @types/babel__generator @types/glob

Some of these packages were used in the previous module (

) will be covered shortly. For more details on the type packages, see the sidenote on typings at the end.

 file that is mostly a copy of the previous 

 script, but with all the logging removed. This logging was only necessary for performing the initial audit.

import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        const hasButtonClassName = node.attributes.find((attribute) => {
          return (
            attribute.type === "JSXAttribute" &&
            attribute.name.type === "JSXIdentifier" &&
            attribute.name.name === "className" &&
            attribute.value.type === "StringLiteral" &&
            attribute.value.value.split(" ").includes("button")
          );
        });

        if (!hasButtonClassName) return;

        // TODO: transform...
      }
    },
  });
});

Running this script won't do or log anything, but it retains the same logic for finding the 

 component. The first step is to reassign the name.

import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import generate from "@babel/generator";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        const hasButtonClassName = node.attributes.find((attribute) => {
          return (
            attribute.type === "JSXAttribute" &&
            attribute.name.type === "JSXIdentifier" &&
            attribute.name.name === "className" &&
            attribute.value.type === "StringLiteral" &&
            attribute.value.value.split(" ").includes("button")
          );
        });

        if (!hasButtonClassName) return;

        // Change the opening element's 
        // name from `button` to `Button`.
        node.name.name = "Button";
      }
    },
  });

  // Convert the mutated AST back into 
  // a source code string.
  const { code } = generate(ast);

  // Overwrite the existing file with 
  // the mutated source code string.
  fs.writeFileSync(file, code);
});

After updating the node's name, the tree can be turned 

 method. This accepts an AST as the input, and will return a string of code that is equivalent to the AST. Finally, the script will overwrite the existing file with the new code string.

Running this should now make a transformation in the Flash app codebase.

 class name. For example, the login component (

). The opening element should now be an uppercase 

The script converted the code into an AST, changed the AST, and then converted the mutated AST back into code.

However, there are still a number of issues:

Let's tackle each of these problems individually. The first three are requirements. The last is optional since the code is still correct, but formatting is lost.

Before updating the script, take a moment to paste the "After" code from above into 

 and explore the tree. Can you see what the problem is?

. The script needs to be updated to also rename the closing element. However, the script is currently only traversing 

 nodes. For the code audit, all the data that we cared about was on this node.

It is possible to reference parent nodes from the path object. Rather, instead of traversing up and down the tree, it's usually easiest to target the highest node in the tree, while also trying to be as specific (low) as possible in the tree. A more technical definition would be to target the closest shared ancestor of the two nodes.

For example, this script needs to modify both the opening and closing elements. They are both properties on the 

 node. In other words, they are both child nodes of the 

 the parent of both nodes we need to transform. This is not always the case. The closest shared ancestor could be anywhere higher in the tree and often is not the parent node. For example, the closed shared ancestor node could be the grandparent of some nodes and the parent of others.

This is a general guideline that is usually a good starting point. The node to target depends on the exact case. For complex transformations, it may even require targeting multiple nodes.

Now the script can be updated to target the 

 node, and update both the opening and closing element names.

import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import generate from "@babel/generator";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    // Find `JSXElement` nodes instead of `JSXOpeningElement` nodes...
    JSXElement({ node }) {
      // Grab the opening and closing elements nodes. 
      // `openingElement` is a `JSXOpeningElement` node, 
      // which was what we were working with earlier.
      const { openingElement, closingElement } = node;

      // The filtering for only `button` elements with a 
      // `button` class name can be updated to rely on 
      // the `openingElement` instead
      // of `node` and reuse the identical logic.
      if (
        openingElement.name.type === "JSXIdentifier" &&
        openingElement.name.name === "button"
      ) {
        const hasButtonClassName = openingElement.attributes.find(
          (attribute) => {
            return (
              attribute.type === "JSXAttribute" &&
              attribute.name.type === "JSXIdentifier" &&
              attribute.name.name === "className" &&
              attribute.value.type === "StringLiteral" &&
              attribute.value.value.split(" ").includes("button")
            );
          }
        );

        if (!hasButtonClassName) return;

        openingElement.name.name = "Button";

        // In addition to updating the opening element's 
        // name, the closing element can also be updated.
        if (closingElement.name.type === "JSXIdentifier") {
          closingElement.name.name = "Button";
        }
      }
    },
  });

  const { code } = generate(ast);

  fs.writeFileSync(file, code);
});

---
title: Mutating an AST
description: Transforming code in place by mutating an AST
privateVideoUrl: "https://fullstack.wistia.com/medias/d35ionn0y3"
---

# Mutating an AST

The script from the previous module read files from the codebase, traversed the source code within those files to find the relevant code (nodes in an AST), and logged that data. With the data, we were able to define and build a component. Now that we have the component, it's time to go through and replace all the `button` elements with our new `Button` component.

There are a few ways this could be approached:

-   **Manually**: go through all the files, look for `button` elements, and replace them with the `Button` component. This is good for complex changes, but only works well in smaller codebases.
-   **Find/Replace**: use a tool that can find then replace pieces of code based on text or a regular expression. This is reasonable for simple cases, and works well in large codebases.
-   **ASTs**: use AST-based tooling. This is ideal for larger codebases and can also handle complex transformations.

All approaches are valid. The scope and complexity of the change can help determine which is the most efficient.

The button transformation needed here is complex. For example, the `className` string needs to be broken up into multiple props, but those props are only required if they're not the same as the default. Since the sample codebase only contains a handful of buttons, it would be realistic to manually replace them all. However, we're going to use AST-based tooling to understand how it can be applied to transform code.

## Getting started

Again, let's start by creating a new sibling directory to the Flash app, so initialize a new `package.json`, and add the necessary dependencies.

```bash
# Create a new directory for the transform script
mkdir transform

# Change into the new transform directory
cd transform

# Initialize a new package.json file
npm init -y

# Add the necessary dependencies
npm i @babel/parser @babel/traverse @babel/types @babel/generator glob ts-node typescript
npm i -D @types/babel__traverse @types/babel__generator @types/glob
```

Some of these packages were used in the previous module (`@babel/parser`, `@babel/traverse`, `glob`, `ts-node`, and `typescript`). The new babel packages (`@babel/types` and `@babel/generator`) will be covered shortly. For more details on the type packages, see the sidenote on typings at the end.

## Creating a transform script

Let's continue by creating a new `transform.ts` file that is mostly a copy of the previous `audit.ts` script, but with all the logging removed. This logging was only necessary for performing the initial audit.

```typescript { actualFilename=./protected/transform/transform-00.ts endChar=991 endLine=36 startChar=0 startLine=1 statedFilename=transform.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.01/protected/transform/transform-00.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        const hasButtonClassName = node.attributes.find((attribute) => {
          return (
            attribute.type === "JSXAttribute" &&
            attribute.name.type === "JSXIdentifier" &&
            attribute.name.name === "className" &&
            attribute.value.type === "StringLiteral" &&
            attribute.value.value.split(" ").includes("button")
          );
        });

        if (!hasButtonClassName) return;

        // TODO: transform...
      }
    },
  });
});
```

Running this script won't do or log anything, but it retains the same logic for finding the `button` elements. Currently, the node's name (`node.name.name`) is `button`, but now we need to change it to our `Button` component. The first step is to reassign the name.

```typescript { actualFilename=./protected/transform/transform-01.ts endChar=1332 endLine=47 startChar=0 startLine=1 statedFilename=transform.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.01/protected/transform/transform-01.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import generate from "@babel/generator";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        const hasButtonClassName = node.attributes.find((attribute) => {
          return (
            attribute.type === "JSXAttribute" &&
            attribute.name.type === "JSXIdentifier" &&
            attribute.name.name === "className" &&
            attribute.value.type === "StringLiteral" &&
            attribute.value.value.split(" ").includes("button")
          );
        });

        if (!hasButtonClassName) return;

        // Change the opening element's 
        // name from `button` to `Button`.
        node.name.name = "Button";
      }
    },
  });

  // Convert the mutated AST back into 
  // a source code string.
  const { code } = generate(ast);

  // Overwrite the existing file with 
  // the mutated source code string.
  fs.writeFileSync(file, code);
});
```

After updating the node's name, the tree can be turned _back_ into source code using the [`@babel/generator`](https://babeljs.io/docs/en/babel-generator) package's `generate` method. This accepts an AST as the input, and will return a string of code that is equivalent to the AST. Finally, the script will overwrite the existing file with the new code string.

Running this should now make a transformation in the Flash app codebase.

```bash
npx ts-node ./transform.ts
```

Look at a file that contained a `button` element that also had the `button` class name. For example, the login component (`src/components/LogIn/LogIn.js`). The opening element should now be an uppercase `Button`.

### Before

```jsx { actualFilename=./protected/button-before-00.jsx endChar=119 endLine=3 startChar=0 startLine=1 statedFilename=button-before.jsx url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.01/protected/button-before-00.jsx }
<button type="submit" className="button button--secondary button--block">
  <span role="img">🔐</span> Log In
</button>
```

### After

```jsx { actualFilename=./protected/button-after-00.jsx endChar=119 endLine=3 startChar=0 startLine=1 statedFilename=button-after.jsx url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.01/protected/button-after-00.jsx }
<Button type="submit" className="button button--secondary button--block">
  <span role="img">🔐</span> Log In
</button>
```

You've created your first codemod! 🎉

The script converted the code into an AST, changed the AST, and then converted the mutated AST back into code.

However, there are still a number of issues:

1.  Only the _opening_ element was renamed so the closing element is still lowercase `button`.
2.  The props are no longer correct since `className` is no longer an option, and many have defaults that aren't required. Recall from implementing the `Button` component that using the `variant` and `block` props instead was done for ease of maintenance and styling consistency.
3.  The `Button` component isn't imported.
4.  The formatting changed in some of the files.

Let's tackle each of these problems individually. The first three are requirements. The last is optional since the code is still correct, but formatting is lost.

## Renaming closing elements

Before updating the script, take a moment to paste the "After" code from above into [AST Explorer](https://astexplorer.net) and explore the tree. Can you see what the problem is?

The `JSXOpeningElement` node is referenced by the `openingElement` property of a `JSXElement`. The `JSXElement` also has a `closingElement` property that references a `JSXClosingElement`.

The problem here is only the `JSXOpeningElement` node's `name` was changed, and not the `JSXClosingElement` node's `name`. The script needs to be updated to also rename the closing element. However, the script is currently only traversing `JSXOpeningElement` nodes. For the code audit, all the data that we cared about was on this node.

It is possible to reference parent nodes from the path object. Rather, instead of traversing up and down the tree, it's usually easiest to target the highest node in the tree, while also trying to be as specific (low) as possible in the tree. A more technical definition would be to target the closest shared ancestor of the two nodes.

For example, this script needs to modify both the opening and closing elements. They are both properties on the `JSXElement` node. In other words, they are both child nodes of the `JSXElement` node.

![example of targeting two nodes' parent](https://d2uusema5elisf.cloudfront.net/courses/practical-abstract-syntax-trees/module_04/lesson_04.01/public/assets/node-to-target-basic-case.png)

In this case, the `JSXElement` node is the closest shared ancestor _and_ the parent of both nodes we need to transform. This is not always the case. The closest shared ancestor could be anywhere higher in the tree and often is not the parent node. For example, the closed shared ancestor node could be the grandparent of some nodes and the parent of others.

![example of more complex node targeting](https://d2uusema5elisf.cloudfront.net/courses/practical-abstract-syntax-trees/module_04/lesson_04.01/public/assets/node-to-target-complex-case.png)

This is a general guideline that is usually a good starting point. The node to target depends on the exact case. For complex transformations, it may even require targeting multiple nodes.

Now the script can be updated to target the `JSXElement` node, and update both the opening and closing element names.

```typescript { actualFilename=./protected/transform/transform-02.ts endChar=1928 endLine=62 startChar=0 startLine=1 statedFilename=transform.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.01/protected/transform/transform-02.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import generate from "@babel/generator";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    // Find `JSXElement` nodes instead of `JSXOpeningElement` nodes...
    JSXElement({ node }) {
      // Grab the opening and closing elements nodes. 
      // `openingElement` is a `JSXOpeningElement` node, 
      // which was what we were working with earlier.
      const { openingElement, closingElement } = node;

      // The filtering for only `button` elements with a 
      // `button` class name can be updated to rely on 
      // the `openingElement` instead
      // of `node` and reuse the identical logic.
      if (
        openingElement.name.type === "JSXIdentifier" &&
        openingElement.name.name === "button"
      ) {
        const hasButtonClassName = openingElement.attributes.find(
          (attribute) => {
            return (
              attribute.type === "JSXAttribute" &&
              attribute.name.type === "JSXIdentifier" &&
              attribute.name.name === "className" &&
              attribute.value.type === "StringLiteral" &&
              attribute.value.value.split(" ").includes("button")
            );
          }
        );

        if (!hasButtonClassName) return;

        openingElement.name.name = "Button";

        // In addition to updating the opening element's 
        // name, the closing element can also be updated.
        if (closingElement.name.type === "JSXIdentifier") {
          closingElement.name.name = "Button";
        }
      }
    },
  });

  const { code } = generate(ast);

  fs.writeFileSync(file, code);
});
```

The updates contain a few key changes. Instead of traversing `JSXOpeningElement` nodes, it's now targeting `JSXElement` nodes. The original logic to filter to only `button` elements was also updated to use the `openingElement`. Finally, the `closingElement` node's name is also being updated. Running this script should now properly convert `button` elements to a `Button` component.

```bash
npx ts-node ./transform.ts
```

:::note

Each time this transform script is run, it will directly transform and rewrite code in the Flash codebase. Assuming the transform is correct, this is the desired outcome. However, it's possible that while testing a transform you'll want to run it on the codebase several times. Before it can be run again, the previous changes will need to be undone. Each change could be manually reverted but an easier way would be to use git in the Flash codebase and run `git reset --hard`, or an equivalent command. This will remove all your uncommitted changes, so make sure any previous work is committed.

Alternatively, you can redownload the latest Flash app from a previous lesson.

:::

The script now properly updates both opening and closing elements.

```jsx { actualFilename=./protected/button-after-01.jsx endChar=120 endLine=4 startChar=0 startLine=1 statedFilename=button-after.jsx url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_04/lesson_04.01/protected/button-after-01.jsx }
<Button type="submit" className="button button--secondary button--block">
  <span role="img">🔐</span> Log In
</Button>
```

## Updating the props

When we implemented the `Button` component in the previous module, defaults were added for some props and some props were replaced with others. So far, we've renamed `button` elements to `Button` components but haven't accounted for any of these other differences in the props. The next step is to update the props and make them valid with the `Button` component.

For props with values that match the new defaults, they can be omitted since they are redundant. For props that were replaced, they need to be converted to the replacement prop and the old prop removed. Lastly, there are some props that were unchanged.

The script can be updated to handle all these props changes. The specific changes and related props are as follows:

Transforming code in place by mutating an AST

Hi! At startup I get this error:TSError: ⨯ Unable to compile TypeScript:transform.ts:25:25 - error TS2533: Object is possibly 'null' or 'undefined'.25                         attribute.value.type === "StringLiteral" &&transform.ts:26:25 - error TS2533: Object is possibly 'null' or 'undefined'.26                         attribute.value.value.split(" ").includes("button")transform.ts:26:41 - error TS2339: Property 'value' does not exist on type 'JSXElement | JSXExpressionContainer | JSXFragment | StringLiteral'.  Property 'value' does not exist on type 'JSXElement'.26                         attribute.value.value.split(" ").includes("button")How can this be corrected?

Artem Alekseenko

Hey Artem, sorry for the delayed response. Could you share the full code snippet? Have you included all the conditional checks on lines 21-27? These checks are required to handle every node type at runtime and required by the TypeScript compiler for it to be able to type check everything properly.

Learn

The newline Guide to Building Your First GraphQL Server with Node and TypeScript

Teach

Amelia Wattenberger

Author of Fullstack D3

Community

Practical Abstract Syntax Trees

Module 1: Introduction

Introduction:

Start Here

Introduction: (2:27)

Course overview

Module 2: Understanding Abstract Syntax Trees (AST)

Lesson 1: (5:55)

What is an Abstract Syntax Tree? Visualizing Code Like a Compiler

Lesson 2: (4:21)

How to View Abstract Syntax Tree Code With AST Explorer

Lesson 3: (1:46)

The Best JavaScript AST Tools - ESLint, Babel, Terser, and More

Lesson 4: (1:03)

Environment setup

Module 3: Working with Abstract Syntax Trees

Lesson 1: (6:29)

How to Generate a JavaScript AST With Babel Plugins

Lesson 2: (7:33)

Traversing an AST With Babel Traverse

Lesson 3: (9:31)

Add Type Safety and Prevent Runtime Errors With AST

Module 4: Code Audits

Lesson 1: (2:35)

Practical code audits

Lesson 2: (15:29)

How to Audit Your Code With AST Programming

Lesson 3: (2:32)

How to Add a Component and Update an AST

Lesson 4: (1:43)

When to Use Abstract Syntax Tree Tooling to Refactor at Scale

Module 5: Codemods

Lesson 1: (21:13)

How to Mutate an AST and Automatically Replace Code Components

Lesson 2: (10:03)

How to Create Codemods Using jscodeshift

Module 6: Linting

Lesson 1: (8:44)

How to Ensure Codebase is Up to Date With Linting Rules

Lesson 2: (5:31)

How to Use ESLint to Create AST Rules With Babel Traverse

Mutating an AST