In this course, we'll start with the fundamentals of abstract syntax trees (ASTs) and learn the basic mental models. This general AST knowledge can be translated to almost any tool that works with ASTs.

## Why this course?

Understanding and using ASTs unlocks the ability to make sweeping changes in a safe and reliable way in any size codebase.

## Course topics

Throughout this course, we'll have converted source code into ASTs, traversed, mutated, and generated ASTs. With these concepts we'll then explore several practical applications including things like code audits (static analysis), code transformations (codemods), and linting.

### Module 1

We'll learn the fundamentals of abstract syntax trees. 

- What is an AST?
- How to explore an AST
- Examples of JavaScript tools that work with ASTs

### Module 2

We'll learn how to work with ASTs.

- How to turn code into an AST
- How to programmatically navigate any AST 
- How to leverage TypeScript to prevent runtime errors

### Module 3

We'll learn how to statically analyze, or "audit" code to understand the state of the codebase using abstract syntax trees.

- An introduction to an example codebase and refactor
- Understanding the state of the current codebase
- When to use an AST-based tool versus doing something manually

### Module 4

We'll learn how to transform, or "codemod" code from one state to another using abstract syntax trees.

- How to make changes to an AST
- How to change ASTs with [jscodeshift](https://github.com/facebook/jscodeshift)
- How to test a code transform

### Module 5

We'll learn how to write rules, or "lint" code using abstract syntax trees.

- How to create rules for code
- How to create custom rules with [ESLint](https://eslint.org)
- How to test a rule

In this course, you'll learn the fundamentals of abstract syntax trees, what they are, how they work, and dive into several practical use cases of abstract syntax trees to maintain a JavaScript codebase.

Practical Abstract Syntax Trees

---
title: How to Mutate an AST and Automatically Replace Code Components
description: Transforming code in place by mutating an AST
privateVideoUrl: "https://fullstack.wistia.com/medias/d35ionn0y3"
code: https://gitlab.com/fullstackio/books/newline-course-apps/spencer-miskoviak-practical-abstract-syntax-trees-app/-/tree/main/code/module_05/lesson_05.01/App?ref_type=heads
---

# Mutating an AST

The script from the previous module read files from the codebase, traversed the source code within those files to find the relevant code (nodes in an AST), and logged that data. With the data, we were able to define and build a component. Now that we have the component, it's time to go through and replace all the `button` elements with our new `Button` component.

There are a few ways this could be approached:

-   **Manually**: go through all the files, look for `button` elements, and replace them with the `Button` component. This is good for complex changes, but only works well in smaller codebases.
-   **Find/Replace**: use a tool that can find then replace pieces of code based on text or a regular expression. This is reasonable for simple cases, and works well in large codebases.
-   **ASTs**: use AST-based tooling. This is ideal for larger codebases and can also handle complex transformations.

All approaches are valid. The scope and complexity of the change can help determine which is the most efficient.

The button transformation needed here is complex. For example, the `className` string needs to be broken up into multiple props, but those props are only required if they're not the same as the default. Since the sample codebase only contains a handful of buttons, it would be realistic to manually replace them all. However, we're going to use AST-based tooling to understand how it can be applied to transform code.

## Getting started

Again, let's start by creating a new sibling directory to the Flash app, so initialize a new `package.json`, and add the necessary dependencies.

```bash
# Create a new directory for the transform script
mkdir transform

# Change into the new transform directory
cd transform

# Initialize a new package.json file
npm init -y

# Add the necessary dependencies
npm i @babel/parser @babel/traverse @babel/types @babel/generator glob ts-node typescript
npm i -D @types/babel__traverse @types/babel__generator @types/glob
```

Some of these packages were used in the previous module (`@babel/parser`, `@babel/traverse`, `glob`, `ts-node`, and `typescript`). The new babel packages (`@babel/types` and `@babel/generator`) will be covered shortly. For more details on the type packages, see the sidenote on typings at the end.

## Creating a transform script

Let's continue by creating a new `transform.ts` file that is mostly a copy of the previous `audit.ts` script, but with all the logging removed. This logging was only necessary for performing the initial audit.

```typescript { actualFilename=./protected/transform/transform-00.ts endChar=991 endLine=36 startChar=0 startLine=1 statedFilename=transform-00.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_05/lesson_05.01/protected/transform/transform-00.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        const hasButtonClassName = node.attributes.find((attribute) => {
          return (
            attribute.type === "JSXAttribute" &&
            attribute.name.type === "JSXIdentifier" &&
            attribute.name.name === "className" &&
            attribute.value.type === "StringLiteral" &&
            attribute.value.value.split(" ").includes("button")
          );
        });

        if (!hasButtonClassName) return;

        // TODO: transform...
      }
    },
  });
});
```

Running this script won't do or log anything, but it retains the same logic for finding the `button` elements. Currently, the node's name (`node.name.name`) is `button`, but now we need to change it to our `Button` component. The first step is to reassign the name.

```typescript { actualFilename=./protected/transform/transform-01.ts endChar=1332 endLine=47 startChar=0 startLine=1 statedFilename=transform-01.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_05/lesson_05.01/protected/transform/transform-01.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import generate from "@babel/generator";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    JSXOpeningElement({ node }) {
      if (node.name.type === "JSXIdentifier" && node.name.name === "button") {
        const hasButtonClassName = node.attributes.find((attribute) => {
          return (
            attribute.type === "JSXAttribute" &&
            attribute.name.type === "JSXIdentifier" &&
            attribute.name.name === "className" &&
            attribute.value.type === "StringLiteral" &&
            attribute.value.value.split(" ").includes("button")
          );
        });

        if (!hasButtonClassName) return;

        // Change the opening element's 
        // name from `button` to `Button`.
        node.name.name = "Button";
      }
    },
  });

  // Convert the mutated AST back into 
  // a source code string.
  const { code } = generate(ast);

  // Overwrite the existing file with 
  // the mutated source code string.
  fs.writeFileSync(file, code);
});
```

After updating the node's name, the tree can be turned _back_ into source code using the [`@babel/generator`](https://babeljs.io/docs/en/babel-generator) package's `generate` method. This accepts an AST as the input, and will return a string of code that is equivalent to the AST. Finally, the script will overwrite the existing file with the new code string.

Running this should now make a transformation in the Flash app codebase.

```bash
npx ts-node ./transform-01.ts
```

Look at a file that contained a `button` element that also had the `button` class name. For example, the login component (`src/components/LogIn/LogIn.js`). The opening element should now be an uppercase `Button`.

### Before

```jsx { actualFilename=./protected/button-before-00.jsx endChar=119 endLine=3 startChar=0 startLine=1 statedFilename=button-before.jsx url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_05/lesson_05.01/protected/button-before-00.jsx }
<button type="submit" className="button button--secondary button--block">
  <span role="img">🔐</span> Log In
</button>
```

### After

```jsx { actualFilename=./protected/button-after-00.jsx endChar=119 endLine=3 startChar=0 startLine=1 statedFilename=button-after.jsx url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_05/lesson_05.01/protected/button-after-00.jsx }
<Button type="submit" className="button button--secondary button--block">
  <span role="img">🔐</span> Log In
</button>
```

You've created your first codemod! 🎉

The script converted the code into an AST, changed the AST, and then converted the mutated AST back into code.

However, there are still a number of issues:

1.  Only the _opening_ element was renamed so the closing element is still lowercase `button`.
2.  The props are no longer correct since `className` is no longer an option, and many have defaults that aren't required. Recall from implementing the `Button` component that using the `variant` and `block` props instead was done for ease of maintenance and styling consistency.
3.  The `Button` component isn't imported.
4.  The formatting changed in some of the files.

Let's tackle each of these problems individually. The first three are requirements. The last is optional since the code is still correct, but formatting is lost.

## Renaming closing elements

Before updating the script, take a moment to paste the "After" code from above into [AST Explorer](https://astexplorer.net) and explore the tree. Can you see what the problem is?

The `JSXOpeningElement` node is referenced by the `openingElement` property of a `JSXElement`. The `JSXElement` also has a `closingElement` property that references a `JSXClosingElement`.

The problem here is only the `JSXOpeningElement` node's `name` was changed, and not the `JSXClosingElement` node's `name`. The script needs to be updated to also rename the closing element. However, the script is currently only traversing `JSXOpeningElement` nodes. For the code audit, all the data that we cared about was on this node.

It is possible to reference parent nodes from the path object. Rather, instead of traversing up and down the tree, it's usually easiest to target the highest node in the tree, while also trying to be as specific (low) as possible in the tree. A more technical definition would be to target the closest shared ancestor of the two nodes.

For example, this script needs to modify both the opening and closing elements. They are both properties on the `JSXElement` node. In other words, they are both child nodes of the `JSXElement` node.

![example of targeting two nodes' parent](https://d2uusema5elisf.cloudfront.net/courses/practical-abstract-syntax-trees/module_05/lesson_05.01/public/assets/node-to-target-basic-case.png)

In this case, the `JSXElement` node is the closest shared ancestor _and_ the parent of both nodes we need to transform. This is not always the case. The closest shared ancestor could be anywhere higher in the tree and often is not the parent node. For example, the closed shared ancestor node could be the grandparent of some nodes and the parent of others.

![example of more complex node targeting](https://d2uusema5elisf.cloudfront.net/courses/practical-abstract-syntax-trees/module_05/lesson_05.01/public/assets/node-to-target-complex-case.png)

This is a general guideline that is usually a good starting point. The node to target depends on the exact case. For complex transformations, it may even require targeting multiple nodes.

Now the script can be updated to target the `JSXElement` node, and update both the opening and closing element names.

```typescript { actualFilename=./protected/transform/transform-02.ts endChar=1928 endLine=62 startChar=0 startLine=1 statedFilename=transform-02.ts url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_05/lesson_05.01/protected/transform/transform-02.ts }
import { parse } from "@babel/parser";
import traverse from "@babel/traverse";
import generate from "@babel/generator";
import * as fs from "fs";
import * as glob from "glob";

const files = glob.sync("../flash-cards/src/components/**/*.js");

files.forEach((file) => {
  const contents = fs.readFileSync(file).toString();

  const ast = parse(contents, {
    sourceType: "module",
    plugins: ["jsx"],
  });

  traverse(ast, {
    // Find `JSXElement` nodes instead of `JSXOpeningElement` nodes...
    JSXElement({ node }) {
      // Grab the opening and closing elements nodes. 
      // `openingElement` is a `JSXOpeningElement` node, 
      // which was what we were working with earlier.
      const { openingElement, closingElement } = node;

      // The filtering for only `button` elements with a 
      // `button` class name can be updated to rely on 
      // the `openingElement` instead
      // of `node` and reuse the identical logic.
      if (
        openingElement.name.type === "JSXIdentifier" &&
        openingElement.name.name === "button"
      ) {
        const hasButtonClassName = openingElement.attributes.find(
          (attribute) => {
            return (
              attribute.type === "JSXAttribute" &&
              attribute.name.type === "JSXIdentifier" &&
              attribute.name.name === "className" &&
              attribute.value.type === "StringLiteral" &&
              attribute.value.value.split(" ").includes("button")
            );
          }
        );

        if (!hasButtonClassName) return;

        openingElement.name.name = "Button";

        // In addition to updating the opening element's 
        // name, the closing element can also be updated.
        if (closingElement.name.type === "JSXIdentifier") {
          closingElement.name.name = "Button";
        }
      }
    },
  });

  const { code } = generate(ast);

  fs.writeFileSync(file, code);
});
```

The updates contain a few key changes. Instead of traversing `JSXOpeningElement` nodes, it's now targeting `JSXElement` nodes. The original logic to filter to only `button` elements was also updated to use the `openingElement`. Finally, the `closingElement` node's name is also being updated. Running this script should now properly convert `button` elements to a `Button` component.

```bash
npx ts-node ./transform-02.ts
```

:::note

Each time this transform script is run, it will directly transform and rewrite code in the Flash codebase. Assuming the transform is correct, this is the desired outcome. However, it's possible that while testing a transform you'll want to run it on the codebase several times. Before it can be run again, the previous changes will need to be undone. Each change could be manually reverted but an easier way would be to use git in the Flash codebase and run `git reset --hard`, or an equivalent command. This will remove all your uncommitted changes, so make sure any previous work is committed.

Alternatively, you can redownload the latest Flash app from a previous lesson.

:::

The script now properly updates both opening and closing elements.

```jsx { actualFilename=./protected/button-after-01.jsx endChar=120 endLine=4 startChar=0 startLine=1 statedFilename=button-after.jsx url=https&#x3A;//gitlab.com/fullstackio/books/practical-ast-and-codemods/module_05/lesson_05.01/protected/button-after-01.jsx }
<Button type="submit" className="button button--secondary button--block">
  <span role="img">🔐</span> Log In
</Button>
```

## Updating the props

When we implemented the `Button` component in the previous module, defaults were added for some props and some props were replaced with others. So far, we've renamed `button` elements to `Button` components but haven't accounted for any of these other differences in the props. The next step is to update the props and make them valid with the `Button` component.

For props with values that match the new defaults, they can be omitted since they are redundant. For props that were replaced, they need to be converted to the replacement prop and the old prop removed. Lastly, there are some props that were unchanged.

Transforming code in place by mutating an AST