RESTful API Documentation with Go and chi docgen Package

Responses (3)

Stefan Wüthrich2 months ago

A great article.
But the example github project seams to be private (asking for credentials when trying to clone) and not visible on Github

Clap
1|3|

An important chore that gets neglected by developers is writing documentation for their RESTful APIs. Often, this task ends up being assigned lower priority than other tasks, such as building a new feature or modifying an existing feature. Although it delivers no immediate tangible value to end users like features, documentation produces intangible value for developers and their companies. By having detailed information about a RESTful API's endpoints, developers can quickly know how to obtain authorization for protected endpoints, access and interact with certain resources, format the data that's required in a request's body, etc. Ultimately, the value in documentation comes from increased developer productivity and saved development time. The more developers the documentation serves, the more value the documentation produces.

Fortunately, you don't have to spend any time or effort to manually build and maintain documentation from scratch. There are open source tools like Swagger that automate the process of designing and generating RESTful API documentation for developers. These tools:

  • Keep the documentation up-to-date. The documentation automatically reflects any changes made to existing endpoints.

  • Ensure the documentation progresses at the same pace as the RESTful API's development.

  • Scale with the RESTful API's codebase. The documentation grows when adding new endpoints to and shrinks when removing old (or possibly deprecated) endpoints from the RESTful API.

  • Provide a beautiful, interactive, web-based UI for developers to explore the RESTful API.

  • Foster an ecosystem of tools that handle other documentation-related tasks like generating API client libraries (SDKs) and server stubs.

Best of all, most of these tools only need, as input, the source code of the RESTful API or a JSON representation of the RESTful API that's compliant with the OpenAPI Specification. Or, these tools can get fed other representations of the RESTful API that are based on alternative API specifications like RAML (RESTful API Modeling Language).

After generating the documentation, all that's left is to host and share the documentation.

If you built your RESTful API with the Go chi library, then you can automatically generate routing documentation with one of its optional subpackages: docgen.

Below, I'm going to show you how to leverage docgen to:

  • Log routing documentation to STDOUT (upon spinning up the RESTful API).

  • Output documentation as pre-formatted Markdown that can be saved to a Markdown (.md) file. It can be placed within the RESTful API's Git repository to communicate important information about the RESTful API.

  • Output JSON and RAML representations of the RESTful API.

Installation and Setup#

To get started, clone the following repository to your local machine:

This repository contains source code for a simple RESTful API built with Go and chi. The router uses a middleware stack that consists of the following middlewares:

  • RequestID - Adds a request ID to the context of the incoming request. The request ID is formatted as the string host.example.com/<base62_random_string>-<request_count>.

  • RealIP - Sets the http.Request's RemoteAddr to the originating IP address of the incoming request (from either the True-Client-IP, X-Real-IP or X-Forwarded-For header).

  • Logger - Logs information like the response status and response time for the incoming request.

  • Recoverer - Recovers from a panic, logs the cause of the panic and, if possible, returns an HTTP 500 status code.

  • URLFormat - Parses the URL extension and adds it to the context of the incoming request (under middleware.URLFormatCtxKey).

Additionally, render.SetContentType(render.ContentTypeJSON) forces the Content-Type of responses to be application/json.

Most of these middlewares are ported from Goji, a minimalistic web framework for Go, and they are built with native Go packages.

The router defines several endpoints for a resource that represents posts:

  • GET / - Verify whether or not the service is up and running ("health check"). Returns the "Hello World!" message

  • GET /posts - Retrieve a list of posts.

  • POST /posts - Creates a post.

  • GET /posts/{id} - Retrieve a single post identified by its id.

  • PUT /posts/{id} - Update a single post identified by its id.

  • DELETE /posts/{id} - Delete a single post identified by its id.

To start up the RESTful API, first install the project's dependencies. Then, compile and execute the code.

Logging Routing Documentation to STDOUT#

docgen lets you print a list of available routes to STDOUT at runtime, like so:

This gives an overview of the resources that can be accessed from this RESTful API.

To log routing documentation to STDOUT, first add the github.com/go-chi/docgen dependency to the list of imported packages and dependencies in main.go, like so:

Note #1: If you encounter the error message cannot use r (variable of type *"github.com/go-chi/chi".Mux) as type "github.com/go-chi/chi/v5".Routes in argument to docgen.JSONRoutesDoc:, then you should change the imported dependency from github.com/go-chi/chi to "github.com/go-chi/chi/v5.

Note #2: If you encounter the error message http: panic serving 127.0.0.1:50114: runtime error: slice bounds out of range [-1:], then you should change the imported dependency from github.com/go-chi/chi/middleware to github.com/go-chi/chi/v5/middleware.

Then, inside of the main() function, call the method docgen.PrintRoutes() after mounting the sub-router defined on the postsResource struct to the main router, like so:

Stop the RESTful API service and re-run the following commands to install the newly added dependency (github.com/go-chi/docgen) and restart the RESTful API service:

Now, the RESTful API's routes get logged to STDOUT.

Outputting Documentation as Pre-Formatted Markdown#

Since documentation should be shared with other developers, let's output the routing information to a pre-formatted Markdown document. Using the docgen.MarkdownRoutesDoc() method, docgen crawls the router to build a route tree. For each route and subroute, docgen collects:

  • The middleware stack that's ran when a request is sent to the route/subroute.

  • The allowed HTTP methods for the route/subroute.

  • The handler function that gets called for each allowed HTTP method.

Then, using this tree, docgen structures and outputs the Markdown as a string. docgen will link each middleware and handler function to its implementation in the source code (the file and line where the implementation can be located).

The docgen.MarkdownRoutesDoc() method accepts two arguments:

  • The router.

  • A set of options for customizing the Markdown. The options must be passed as a docgen.MarkdownOpts struct, which gets passed by value. This struct contains the following fields:

    • Intro (string) - Introductory text that's placed below the title of the Markdown document.

    • ProjectPath (string)- The base Go import path of the project.

    • ForceRelativeLinks (bool) - If you plan on not hosting the project on GitHub, then should the links in the Markdown document still be relative?

    • URLMap (map[string]string) - A mapping between vendored dependencies and their upstream sources.

Note: Any fields omitted from the struct are zero-valued.

Unlike logging the route documentation to STDOUT, the Markdown should not be generated every time the RESTful API service starts up, especially if you want to write the stringified Markdown to a separate file. Instead, let's generate the Markdown only when the user passes a specific flag to the go run command.

First, let's add the flag and errors packages to the list of imported packages and dependencies in main.go, like so:

Just before the main() function, declare a string flag docs with a default value of an empty string (second argument) and a short description (third argument).

Note: This flag will support three values: markdown, json and raml. -docs=markdown will output the routing information to a pre-formatted Markdown document, while -docs=json and -docs=raml will output JSON and RAML representations of the RESTful API respectively. We will implement the json and raml flag values in the next section of this tutorial.

At the start of the main() function body, call the flag.Parse() method to parse the command-line flags into any flags defined before the main() function.

At the end of the main() function body, after calling the docgen.PrintRoutes() method, check if the docs flag has been set to markdown. If so, then call the docgen.MarkdownRoutesDoc() method to generate the Markdown and write it to a routes.md file, like so:

Note #1: f, err := os.Create("routes.md") does not follow the same pattern as the other conditional statements (of having a declaration precede conditionals). Otherwise, f.Close() would cause the error undefined: f since f would only be scoped to its corresponding conditional statement's branches. For the statement to follow the same pattern as the other conditional statements, you would need to declare the variables f and err outside and replace the short declaration with a standard assignment.

Note #2: Even if the routes.md file does not exist, os.Remove will return an error remove routes.md: no such file or directory. Therefore, to prevent this error from causing the program to exit, check that the error is not an os.ErrNotExist error via the errors.Is() method.

Now, add a new rule to the Makefile that compiles and executes the code with the -docs=markdown flag.

(Makefile)

Also, add the following code to the top of the Makefile so that docgen can detect the GOPATH environment variable when the go run command is executed via a make command.

(Makefile)

Note: Without this, you will encounter the error message ERROR: docgen: unable to determine your $GOPATH.

When you run the make command with the target gen_docs_md, Go generates the routing documentation in Markdown and outputs it to a routes.md file:

Below is what the Markdown file should look like. The bulleted middleware and handler functions link directly to their implementation in the source code (file and line).

(routes.md)

Outputting JSON and RAML Representations of the RESTful API#

Having JSON and RAML representations of a RESTful API gives other software and tools the ability to understand the RESTful API's endpoints, middleware stack, etc. These representations can be consumed to create beautiful, custom UIs for displaying the documentation, build testing and scaffolding tools, etc.

The APIs provided by docgen for generating JSON and RAML representations of the RESTful API are quite different. Therefore, let's start with the much simpler task of generating the JSON representation of the RESTful API.

In the main() function, add another conditional branch (else if statement) to the conditional statement *docs == "markdown" that checks if the docs flag is set to json.

If so, then call the docgen.JSONRoutesDoc() method to generate the JSON representation and write it to a routes.json file, like so:

Now, add a new rule to the Makefile that compiles and executes the code with the -docs=json flag.

(Makefile)

When you run the make command with the target gen_docs_json, Go generates the JSON representation and outputs it to a routes.json file:

Below is what the JSON file should look like. The JSON data organizes the routes in a hierarchical order and contains information like the different HTTP methods allowed for a given route, the handler function and middleware stack that's called for a given endpoint, the location of the handler function in the source code, any comments written above the handler function's definition, etc.

(routes.json)

Note: Although the JSON representation of the RESTful API that's generated by docgen does not fully adhere to the OpenAPI specification, you can still apply a few transformations to make it compliant and work with tools like Swagger UI.

For the RAML representation, we need to add the strings package and two more dependencies to the list of imported packages and dependencies in main.go:

Then, in the main() function, add another conditional branch (else if statement) to the conditional statement *docs == "markdown" that checks if the docs flag is set to raml.

If so, then perform the following steps to generate the RAML representation of the RESTful API:

  1. Declare a variable named ramlDocs and assign to it a pointer to the struct raml.RAML. This struct can have the following fields:

  • Title (string) - A label for the API.

  • BaseUri (string) - The base URI for all resource URIs.

  • Protocols ([]string) - The protocols supported by the API (e.g., HTTP and HTTPS).

  • MediaType (string) - The media type that's used for the bodies/payloads of requests and responses (e.g., application/json).

  • Version (string) - The version of the API formatted as v<version_number> (e.g., v1.0).

  • Documentation - An optional list of documents that provide extra guidance and legal/technical details for using the API.

    • Title - A label for the document.

    • Content - A description of what the document contains.

  1. Traverse the router tree via the chi.Walk() method. For each visited API endpoint, chi.Walk() calls a function that provides the endpoint's HTTP method, route, middleware stack and handler function. For the function signature, check out the WalkFunc type defined in the source code of the Go chi library.

  2. In the function, obtain more information about the endpoint's handler function, such as its name and the comments above its implementation in the source code, by passing it to the docgen.GetFuncInfo method. With this information, create a RAML resource with the raml.Resource struct and add it to ramlDocs via the ramlDocs.Add() method. In the outputted RAML, the key that identifies a resource is the resource's relative URI (e.g., /posts). The raml.Resource struct can have the following fields:

  • DisplayName (string) - A label for the resource.

  • Description (string) - A description of the resource.

  • Responses (Responses) - A mapping of response status codes to examples of possible response bodies.

  • Body (Body) - A possible response body.

  • Is ([]string) - A list of resource traits that can be applied to the resource (e.g., secured, rate limited, etc.).

  • Type (string) - A data type that describes what the resource represents (e.g., user, post, etc.).

  • SecuredBy ([]string) - A list of security schemas that can be applied to the endpoints declared for the resource (e.g., OAuth 1.0, OAuth 2.0, etc.).

  • UriParameters ([]string) - A list of URI parameters that the endpoints declared for the resource supports.

  • QueryParameters ([]string) - A list of query parameters that the endpoints declared for the resource supports.

  1. Once chi.Walk() finishes traversing the router tree, and all of the available resources have been added to ramlDocs, serialize ramlDocs into a valid YAML document via the yaml.Marshal() method. Store the document in a variable named raml.

  2. Write raml to a RAML (.raml) file.

Note: The fields in the raml.RAML struct come directly from the RAML specification, and they describe some of the basic aspects of the RESTful API like its title and version. This information ends up in the root section of the outputted RAML document. If you are not familiar with RAML, then please visit the RAML documentation.

Now, add a new rule to the Makefile that compiles and executes the code with the -docs=raml flag.

(Makefile)

Stop the RESTful API service and install the newly added dependencies.

When you run the make command with the target gen_docs_raml, Go generates the RAML representation and outputs it to a routes.raml file:

Below is what the RAML file should look like. Notice how docgen organizes the routes in a similar hierarchical order like what's seen in the JSON representation of the RESTful API, but in a YAML-based format.

Generating Documentation UI for the RAML Representation of the RESTful API#

To turn this RAML file into an interactive, documentation UI that can be shared with others, let's use the raml2html HTML documentation generator.

You can install raml2html globally via npm and run its CLI tool to generate the documentation UI from RAML. However, if you do not want to install raml2html globally on your local machine, then you can also run a Dockerfile that containerizes raml2html.

Run the following docker command to run the raml2html CLI tool in a new container, giving it the routes.raml file as an input and having it output a routes.raml.html file.

Note: If you encounter the error message docker: Error response from daemon: Mounts denied: The path <full_project_directory_path> is not shared from the host and is not known to Docker. You can configure shared paths from Docker -> Preferences... -> Resources -> File Sharing., then you should add the full path of the project directory (whatever gets printed by the pwd) to the list of directories that can be bind mounted into Docker containers. After that's completed, open a new terminal session and re-run the docker command.

Once the routes.raml.html file is generated (located in the root of the project directory), open this file in a browser to see the documentation UI.

If you find yourself stuck at any point while working through this tutorial, then feel free to visit the main branch of this GitHub repository here for the code.

Next Steps#

Explore and try out other automated documentation solutions.

If you want to learn more advanced back-end web development techniques with Go, then check out the Reliable Webservers with Go course by Nat Welch, a site reliability engineer at Time by Ping (and formerly a site reliability engineer at Google), and Steve McCarthy, a senior software engineer at Etsy.

Sources#


Clap
1|3