Tutorials on Middleware

Accessing an e-mail account anywhere in the world on any device requires authenticating yourself to prove the data associated with the account (e.g., e-mail address and inbox messages) actually belongs to you. Often, you must fill out a login form with credentials, such as an e-mail address and password, that uniquely identify your account. When you first create an account, you provide this information in a sign-up form. In some cases, the service sends either a confirmation e-mail or an SMS text message to ensure that you own the supplied e-mail address or phone number. Because it is highly likely that only you know the credentials to your account, authentication prevents unwanted actors from accessing your account and its data. Each time you log into your e-mail account and read your most recent unread messages, you, and like many other end users, don't think about how the service implements authentication to protect/secure your data and hide your activity history. You're busy, and you only want to spend a few minutes in your e-mail inbox before closing it out and resuming your day. For developers, the difficulty in implementing authentication comes from striking a balance between the user experience and the strength of the authentication. For example, a sign up form may prompt the user to enter a password that contains not only alphanumeric characters, but also must meet other requirements such as a minimum password length and containing punctuation marks. Asking for a stronger password decreases the likelihood of a malicious user correctly guessing it, but simultaneously, this password is increasingly more difficult for the user to remember. Keep in mind that poorly designed authentication can easily be bypassed and introduce more vulnerabilities into your application. In most cases, applications implement either session-based or token-based authentication to reliably verify a user's identity and persist authentication for subsequent page visits. Since Go is a popular choice for building server-side applications, Go's ecosystem offers many third-party packages for implementing these solutions into your applications. Below, I'm going to show you how to integrate JWT authentication within a Go and chi application with the chi jwtauth middleware. Let's imagine the following scenario. Within your e-mail inbox, you are asked to re-enter your e-mail address and password on every single action you take (e.g., opening an unread e-mail or switching to a different inbox tab) to continuously verify your identity. This implementation could be useful in the context of accidentally leaving your e-mail inbox open on a publicly-shared library computer when you have to step out to take a phone call. However, if the login credentials are sent over a non-HTTPS connection, then the login credentials are susceptible to a MITM (man-in-the-middle) attack and can be hijacked. Plus, it would result in a frustrating user experience and immediately drive users away to a different service. Traditionally, to persist authentication, an application establishes a session and saves an http-only cookie with this session's ID inside the user's browser. Usually, this session ID maps to the user's ID, which can then be used to fetch the user's information. If you have ever built an Express.js application with the authentication middleware library Passport and session middleware library express-session , then you are probably familiar with the connect.sid http-only cookie, which is a session ID cookie, and managing sessions with Redis . In Redis, the connect.sid cookie's corresponding key is the session ID (the substring proceeding s%3A and preceding the first dot of this cookie's value) prefixed with sess: , and its value contains information about the cookie and user authenticated by Passport. When a user sends an authentication request (via the standard username/password combination or an OAuth 2.0 provider such as Google / Facebook / Twitter ), Passport determines which of these authentication mechanisms ("strategies") to use to process the request. For example, if the user chooses to authenticate via Google, then Passport uses GoogleStrategy , like so: The done function supplies Passport with the authenticated user. To avoid exposing credentials in subsequent requests, the browser uses a unique cookie that identifies the user's session. Passport serializes the least amount of information that's required to map the user to the session. Often, the user's ID gets serialized. By serializing as little information as needed, this means there is less data stored in the user's session. Upon receiving a subsequent requests, Passport deserializes the user's ID (serialized via serializeUser ) into an object with the user's information, which allows it to be up to date with any recent changes. Whenever an Express.js route needs to access this information, it can via the req.user object. With session-based authentication, authentication is stateful because the server persists/tracks the session (either within the server's internal memory or an in-memory data store like Redis or Memcached). With token-based authentication, authentication is stateless . With tokens, nothing needs to be persisted on the server-side, and the server doesn't need to fetch the user's information on every subsequent request. One of the most popular token standards is JSON Web Token (JWT). JWTs are used for authorization, information exchange and verifying the user's authentication. Instead of creating a session, the server creates a cryptographically-signed JWT and saves an http-only cookie with this token inside of the user's browser, which allows the JWT to automatically be sent on every subsequent request. If the JWT is saved in plain memory, then it should be sent in the Authorization header using the Bearer authentication scheme ( Bearer <token> ). A JWT consists of three strings encoded in Base64URL : These strings are concatenated together (separated by dots) to form a token. Example : The following is a simple JWT, which follows the format <BASE64_URL_HEADER>.<BASE64_URL_PAYLOAD>.<BASE64_URL_SIGNATURE> : eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c Constructing JWTs is relatively straight-forward. Decoding a JWT is also relatively straight-forward. Try out different signing algorithms, adding scope: [ "admin", "user" ] to the payload or modifying the secret in the JWT debugger . Note : Since a JWT is digitally signed, its content is protected from tampering. Tampering invalidates the token. Having sensitive data in the payload or header requires the JWT to be encrypted. It is recommended to first sign the JWT and then encrypt it . Referring back to the previous Express.js and Passport example, we can remove both Redis, the session middleware and the serialization/deserialization logic (relies on sessions), and then add the Passport JWT strategy passport-jwt for authenticating with a JWT. We no longer have to devote any backend infrastructure/resources to managing sessions with the introduction of token-based authentication via JWT. This will significantly reduce the number of times we need to query the database for the user's information. Like any other authentication method, token-based authentication comes with its own set of unique problems. For example, when we store the token in a cookie, this cookie is sent on every request (bound to a single domain), even those that don't require the user to be authenticated. Although this cookie is stored with the HttpOnly attribute (inaccessible to JavaScript), it is still susceptible to a Cross-Site Request Forgery attack, which happens when a third-party website successfully sends a request to a service without the user's explicit consent due to cookies (those set by the service's server) being sent on all requests to that service's server. If you're running an online banking service, and one of your users is authenticated and visits a malicious website that sends the request POST https://examplebankingservice.com/transfer when they click on a harmless-looking button, then money will be transferred from the user's bank account since their valid token is sent with the request. To mitigate this vulnerability, set the token's cookie SameSite attribute ( sameSite: "lax" or sameSite: "strict" depending on your needs) and include a CSRF token specific to each user of your service in case of malicious subdomains. It should be set as a hidden form field in forms that send requests to protected endpoints upon being submitted, and your service should regenerate a new CSRF token for the user upon them logging in. This way, malicious websites cannot send requests to protected endpoints unless they also know that specific user's CSRF token. Note : By default, the latest versions of some modern browsers already treat cookies without the SameSite attribute as if this attribute was set to Lax . Setting the SameSite attribute of a cookie to Strict restricts a cookie to its originating website only and prevents cookies from being sent on any cross-site request or iframe. Setting the SameSite attribute of a cookie to Lax causes the same behavior as Strict , but relaxes the cross-site request restriction to target only POST requests. The alternative is to store the token in localStorage , but this is not recommended because localStorage is accessible by any JavaScript code running on your website. Therefore, it is susceptible to a Cross-Site Scripting attack, which allows unwanted JavaScript code to be injected into and executed within your website. Common attack vectors for XSS are passing unsanitized user input directly to eval and appending unsanitized HTML (contains a <script /> tag with malicious code). Unlike sessions, an individual JWT cannot be forcefully invalidated when security concerns arise. Rather, there are approaches that can be taken to invalidate a JWT, such as... Fortunately, supporting JWT authentication in a Go and chi application is made easy with the third-party jwtauth library. Similar to the Express.js and Passport example, jwtauth validates and extracts payload information from a JWT for route handlers via several pre-defined middleware handlers ( jwtauth.Verifier and jwtauth.Authenticator ) and context respectively. To demonstrate this, let's walkthrough a simple login flow: Inside of a Go file, scaffold out the routes using the chi router library. This application involves only four routes: ( main.go ) Let's think about these routes in-depth. When the user logs in, the navigation bar should no longer display a "Log In" link. Instead, the navigation bar should display the user's username as a link, which opens the user's "Profile" page when clicked, and a "Log Out" link. This means that all pages that display the navigation bar should be aware of whether or not the user is logged in, as well as the identity of the user. Let's group the GET / , GET /login and GET /profile endpoints together via the r.Group method, and then execute the middleware handler jwtauth.Verifier to seek, verify and validate the user's JWT token. This handler accepts a pointer to a JWTAuth struct, which is returned by the jwtauth.New method. Essentially, this method creates a factory for generating JWT tokens using a specified algorithm and secret (an additional key must be provided for RSA and ECDSA algorithms). The POST /login and POST /logout endpoints can be grouped together to establish them as routes that don't require a JWT token. Behind-the-scenes, jwtauth.Verifier automatically searches for a JWT token in an incoming request in the following order: Once the JWT token is verified, it is decoded and then set on the request context. This allows subsequent handlers to have direct access to the payload claims and the token itself. When the user submits a login form, their credentials are sent to the endpoint POST /login . It's corresponding route handler checks if the credentials are valid, and when they are, the handler generates a token that encodes parts of the user's information (i.e., their username) as payload claims via a MakeToken function and stores the token cookie within the user's browser, all before redirecting the user to their "Profile" page. Note : Underscores indicate placeholders for unused variables. For this simplicity's sake, we're going to accept any username and password combination as long as each is at least one character long. When the user logs out, this token cookie needs to be deleted. To delete this cookie, set its MaxAge to a value less than zero. After this cookie is deleted, redirect the user back to the homepage. Although the GET / , GET /login and GET /profile endpoints rely on the jwtauth.Verifier middleware, they each need to be grouped individually (not together) to add custom middleware to account for these scenarios: When rendering the webpages via data-driven templates , we need to extract the user's username from the JWT token's payload, which we encoded via the MakeToken function, to display it within the navigation bar. The payload's claims can be accessed from the request's context. Once the templates are parsed and prepared via the template.ParseFiles and template.Must methods respectively, apply these templates ( tmpl ) to the page data data via the ExecuteTemplate method. The second argument of ExecuteTemplate method names the root template that contains the other parsed templates (partials). The output is written to the ResponseWriter , which will render the result as a webpage. Note : If building a service, such as a RESTful API, that requires a 401 response to be returned on protected routes that can only be accessed by an authenticated user, use the jwtauth.Authenticator middleware. Finally, spin up the server by running the go run . command. Within a browser, visit the application at http://localhost:8080 and open the browser's developer console. Observe how the browser sets and unsets the cookie when you log in and out of the application, and watch as the user's username gets extracted from the JWT and displayed in the navigation bar. If you find yourself stuck at any point while working through this tutorial, then feel free to visit the main branch of this GitHub repository here for the code. Explore and try out other token standards for authentication. If you want to learn more advanced back-end web development techniques with Go, then check out the Reliable Webservers with Go course by Nat Welch, a site reliability engineer at Time by Ping (and formerly a site reliability engineer at Google), and Steve McCarthy, a senior software engineer at Etsy.

Disclaimer - If you are unfamiliar with writing a simple unit test for a route handler of a Go and chi RESTful API, then please read this blog post before proceeding on. Go's testing library package provides many utilities for automating the testing of Go code. To write robust tests for Go code, you must already understand how to write a basic Go test suite that contains several TestXxx functions. Writing tests for code, especially within the context of test-driven development (TDD), prioritizes the code's correctness over the code's flexibility to adapt to new/updated requirements. The guarantee of less unexpected regressions offsets the upfront cost of spending more time to write tests alongside application code. In a fast-paced, high-pressure environment, it can be difficult to convince other team members and stakeholders of the value in testing code when time is an extremely limited resource. Another factor that must be considered is the amount of code covered by the tests. If the tests cover only a small percentage of the application code (or a small subset of use cases), then the benefits of having these tests probably won't outweigh the benefits of adding new features or improving existing features. Plus, anytime you decide to refactor the application code, you will also have to update the corresponding tests to reflect these changes. When time is so valuable, the time spent on writing tests could have been spent elsewhere. Therefore, to fully benefit from tests, you must write enough tests such that they cover a large percentage of the application code. If a RESTful API exposes multiple endpoints, then testing a single route handler won't bring much value for the time spent writing it. Testing RESTful APIs built with Go and chi requires testing not only all of the route handlers, but also, the chi /custom middleware handlers. Let's write tests for... Clone a copy of the Go and chi RESTful API (with a basic test for a single endpoint) from GitHub to you machine: This RESTful API specifies five endpoints for performing operations on posts: If you would like to learn how to build this RESTful API, then please visit this blog post . Run the following command to install the project's dependencies: Note : If you run into installation issues, then verify that the version of Go running on your machine is v1.16 . There is a single test within the routes/posts_test.go file: TestGetPostsHandler . It tests the route handler for the GET /posts endpoint, and it mocks out the GetPosts function called by the route handler to avoid sending a network request to the JSONPlaceholder API when executing tests. Compared to testing route handlers for GET requests, testing route handlers for other HTTP methods (e.g., POST and PUT ) involves slightly more code to test requests that carry data in the form of an optional request body: Let's write a test for the POST /posts endpoint. Before starting, we must first review the route handler for this endpoint ( Create ) in the routes/posts.go file. ( routes/posts.go ) To test the route handler Create , we must first mock out the CreatePost function to avoid sending a network request to the JSONPlaceholder API when executing tests. CreatePost accepts a request body and returns an HTTP response and error (if encountered). Therefore, the mock function must follow the same function signature as CreatePost , like so: CreatePost is defined on type JsonPlaceholderMock . The request body must contain the following information: Since this request body (JSON data) must be passed as type io.ReadCloser to CreatePost , it must be read into a buffer, converted into a byte slice and then decoded into a Go struct so the data can be accessed normally. When a POST /posts request is sent to the JSONPlaceholder API, it returns a response that contains the newly created post. This post contains the exact same information as the request body with an additional Id field. This Id field is set to 101 to imply that a new post was added since there is a total of 100 posts, and each post has an Id field set to 1 , 2 , 3 , etc., up to 100 . The JSONPlaceholder API doesn't actually create this new resource, but rather, fakes it. So anytime you send a request to POST /posts , the response returned will have a newly created post with an Id field set to 101 . In our mock function, let's create this dummy data to send back in the response. Encode this struct to JSON via the json.Marshal method and return it within the body of a HTTP response. This HTTP response should return with a 200 status code to indicate a successful request. Note : This HTTP response returned must adhere to the http.Response struct , which accepts an integer value for its StatusCode field and a value of type io.ReadCloser for its Body field. The ioutil.NopCloser method returns a ReadCloser that wraps the Reader (in this case, bytes.NewBuffer(respBody) , which prepares a buffer to read respBody ) with a no-op Close method, which allows the Reader to adhere to the ReadCloser interface. Putting it altogether... With the mock function now implemented, let's write the test for the route handler Create for POST /posts . Start by naming the unit test TestCreatePostHandler , like so... Then, write this test similar to the TestGetPostsHandler test, but with several adjustments to account for it testing a POST route handler: Putting it altogether... To test the chi logger middleware handler ( middleware.Logger ), we must first understand the implementation details of this handler function: ( go-chi/chi/middleware/logger.go ) The middleware.Logger function only calls a package-scoped function, DefaultLogger , which logs information for each incoming request. ( go-chi/chi/middleware/logger.go ) This package-scoped function accepts a handler function and returns a handler function. Therefore, we can write a test that involves passing a simple route handler to middleware.DefaultLogger and calling middleware.DefaultLogger 's ServeHTTP method with a response recorder and created request: Try adding more endpoints and writing tests for those endpoints' route handlers.

The Go programming language is designed to address the types of software infrastructure issues Google faced back in late 2007. At this time, Google predominantly built their infrastructure using the programming languages C++, Java and Python. With a large codebase and having to adapt to rapid technological changes in both computing and key infrastructure components (e.g., multi-core processors, computer networks and clusters), Google found it difficult to remain productive within this environment using these languages. Not only does Go's design focus on improving the productivity of developers/engineers, but Go's design incorporates the best features of C++, Java and Python into a single statically-typed, compiled, high-performance language: Using Go and a lightweight router library, such as chi , writing and launching a simple RESTful API service requires little time and effort, even if you have no experience with Go. chi 's compatibility with the Go standard library package net/http , optional subpackages (middleware, render and docgen) and consistent API allows you to compose mantainable, modular services that run fast. Plus, these services can be deployed to and thrive in a production environment . If you have previously built RESTful APIs using Node.js and a minimal web application framework, such as Express.js , then you will find this to be quite similar! Below, I'm going to show you: To install Go, visit the official Go downloads page and follow the instructions. If you want your machine to support multiple versions of Go, then I recommend installing GVM (Go Version Manager) . Note : This installation walkthrough will be for Mac OS X systems. Run the following command to install GVM to your machine: Either... A.) Close the current terminal window and open a new terminal window. Or... B.) Run the source command mentioned in the output to make the gvm executable available to the current terminal window. Run the following command to install the latest stable version of Go (as of 02/21, v1.16): Notice how the installation fails. The following error message is printed within the output: If we check the contents of the log file /Users/kenchan/.gvm/logs/go-go1.16-compile.log , then the actual cause of the failure will be shown: Because the go executable could not be found, and GVM assumes that this go executable should be Go v1.4, let's install Go v1.4 from binary: Apparently, there is an known issue with installing Go v1.4 on certain versions of Mac OS X. Instead, let's install Go v1.7.1 from binary: Then, set the go executable to Go v1.7.1: Set the environment variable $GOROOT_BOOTSTRAP to the root of this Go installation to build the new Go toolchain, which will provide the necessary tools to compile ( build ), format ( fmt ), run ( run ) source code, download and install packages/dependencies ( get ), etc.: Finally, let's install the latest stable version of Go and set the go executable to it: Note : The --default option sets this version of Go as the default version. Whenever you open a new terminal window and execute go version , this default version will be printed. Create a new project directory named "go-chi-restful-api": This RESTful API service will offer the following endpoints for accessing/manipulating "posts" resources: A post represents an online published piece of writing. It consists of a title, content, an ID and an ID of the user who authored the post: These endpoints will interface directly with JSONPlaceholder , a fake API for testing with dummy data. For example, when it receives a GET /posts request from a user, our Go RESTful API will check for this route and execute the route handler mapped to it. This route handler will send a request to https://jsonplaceholder.typicode.com/posts and pipe the response of that request back to the user. This project will be comprised of two Go files: Create these files within the root of the project directory: ( main.go ) A Go source file begins with a package clause. All source files within the same folder should have the same package name. main happens to be a special package name that declares the package outputs an executable rather than a library and has a main function defined within it. This main function is executed when we run the Go program. This program imports three standard library packages and one third-party package: Inside of the main function... ( posts.go ) This source file contains the sub-router for the /posts route and its sub-routes ( /posts/{id} ). Here, we import three standard library packages and one third-party package: An empty struct postsResources is defined to namespace all functionality related to "posts" resources. func (rs postsResource) Routes() chi.Router defines the Routes method on the struct postsResource . This method returns an instance of the chi router that handles the /posts route and its sub-routes. Recall previously how we attached this sub-router to the parent router in main.go : Note : A struct is a typed collection of fields. In this Routes function, several endpoints for /posts and its sub-routes are defined on this sub-router. Each method call on the r sub-router instance corresponds to a single endpoint: Along with the Routes function, each of these request handler functions is also defined on the struct postsResource . Let's explore the body of a request handler, List , to understand how request handlers work. When the user sends a GET request to /posts , the request handler for this endpoint sends a GET request to https://jsonplaceholder.typicode.com/posts via the http.Get method to fetch a list of posts. If an error is encountered while fetching this list of posts ( err is not nil ), then the http.Error method replies to the original request with the err object's message and a 500 status code ( http.StatusInternalServerError ). If the list is fetched successfully, then close the body of this response ( resp ) to avoid a resource leak. Set the "Content-Type" header of the response ( w , the one to send back to the user) to "application/json" since the posts data is in a JSON format. To extract the list of posts from resp , copy resp.Body to the w buffer with io.Copy , which uses a fixed 32 kB buffer to copy chunks of up to 32 kB at a time. Then, send this response with the retrieved posts back to the user. Once again, if an error is encountered, then reply to the original request with an error message and a 500 status code. Note : _ is commonly used to denote an unused variable. It serves as a placeholder. When sending a POST/PUT request to the RESTful API service, the body of this request can be accessed via r.Body . When the user sends a GET request to /posts/1 , the router must first pass the request through the PostCtx middleware handler. This middleware handler... In a request handler, values from the request's Context can be accessed by the value's corresponding key. So to get the URL parameter we extracted in the previous step and set as a Context value with the key id , call r.Context().Value("id") . Because both keys and values are of type interface{} in Context , cast this URL parameter value as a string . The http package provides shortcut methods for sending GET and POST requests via the Get and Post methods respectively. However, for PUT and DELETE requests, use the newRequest method to create these requests. You must explicitly pass the HTTP method to newRequest and manually add request headers. client.Do sends the HTTP request created by newRequest and returns an HTTP response. Once again, if an error is encountered, then reply to the original request with an error message and a 500 status code. We could have used io.ReadAll to extract the response body, like so... However, io.ReadAll is inefficient compared to io.Copy because it loads the entirety of the response body into memory. Therefore, if the service must handle a lot of users simultaneously, and the size of resp.Body is large (i.e. contents of a file), then it may run out of available memory and eventually crash. To run install RESTful API service, first install all required third-party dependencies. In this case, there is only one third-party dependency to install: github.com/go-chi/chi . Let's add the project's code to its own module using the following command: If you have previously worked with Node.js, then you can think of go.mod as Go's package.json . By having the project's code in its own module, the project's dependencies can now be tracked and managed. Run the following command to install the project's dependencies and generate a go.sum file that contains checksums of the modules that this project depends on: These checksums are used by Go to verify the integrity of the downloaded modules. If you have previously worked with Node.js, then you can think of go.mod as Go's package.json and go.sum as Go's package-lock.json . Run the following command to run the command: If you would like to run the service on a different port, then set the environment variable PORT to a different port number: Test the service by sending a POST request to /posts : It works! Try sending requests to the other endpoints. chi offers a lot of useful, pre-defined middleware handlers via its optional middleware package. Visit this link for a list of these middleware handlers. One such middleware handler is Logger , which prints on every incoming request: Let's import the github.com/go-chi/chi/middleware package and add the Logger middleware to main.go : ( main.go ) In a separate terminal tab/window, send the same POST request to /posts : Information about this request is logged to the terminal. Click here for a final version of the RESTful API. Try adding more routes to this RESTful API! JSONPlaceholder provides five other resources (comments, albums, photos, todos and users) that you can create endpoints for.