The amount of time and effort a developer dedicates towards writing a function depends on the details they choose to focus on: coding conventions, structure, programming style, etc. Suppose a group of developers is presented a high-level prompt to write the same function: given some input, return some output. For example, given a list of numbers, return a sorted list of numbers. The actual implementation of the function is left entirely to the discretion of the developer. A quick, mathematical way to evaluate each developer's implementation of this function, without any additional code, is by its 

. Particularly, knowing each implementation's Big-O complexity tells us how it might perform in the worst case scenario, commonly when the size of the input is very large.  

However, time complexity fails to account for the hardware the function is executed upon, and it does not provide any tangible, quantifiable metrics to base decisions on. Metrics such as 

 assign real numerical values to the performance of a function. By adding 

, developers can leverage these metrics to better inform them on how to improve their code. The Go programming language has a 

 in its built-in, standard library package 

. To benchmark code in Go, define a function with a name prefixed with 

 (followed by a capitalized segment of text) and accepts an argument of struct type 

, which contains methods and values for determining the number of iterations to run, running multiple benchmarks in parallel, timing execution times, etc. 

import (
  "testing"
)

func BenchmarkSumFunc(b *testing.B) {
  for i := 0; i < b.N; i++ {
    Sum(100, i)
  }
}

: The structure of a benchmark is similar to the structure of a test. Replace 

 iterations. Here, the benchmark runs for one billion iterations, which allows the benchmark function to reliably time and record each execution. Once the benchmark is completed, the results of this benchmark and the CPU of the machine running this benchmark are outputted to the terminal. On average, each iteration ran 

git clone --basic-tests https://github.com/newline-sandbox/go-chi-restful-api.git

This RESTful API specifies five endpoints for performing operations on posts:

If you would like to learn how to build this RESTful API, then please visit this 

 branch, a simple unit test is already provided in the 

 file. Because benchmarks must be placed in 

 files, let's place the benchmarks for the 

Run the following command to install the project's dependencies:

: If you run into installation issues, then verify that the version of Go running on your machine is 

Run the following command to execute the unit tests within the 

 file. Let's name the benchmark function 

func BenchmarkGetPostsHandler(b *testing.B) {
  // ...
}

// ...

func BenchmarkGetPostsHandler(b *testing.B) {
  b.Run("Endpoint: GET /posts", func(b *testing.B) {
    GetPosts = (&JsonPlaceholderMock{}).GetPosts
    r, _ := http.NewRequest("GET", "/posts", nil)
    w := httptest.NewRecorder()
    handler := http.HandlerFunc(PostsResource{}.List)

    b.ReportAllocs()
    b.ResetTimer()

    for i := 0; i < b.N; i++ {
      handler.ServeHTTP(w, r)
    }
  })
}

 iterations with each iteration running, on average, 

 ns. This represents the average time it took for each 

) call to complete. Each iteration involved the allocation of, on average, 

 bytes of memory. Memory was allocated, on average, 

 loop forces the benchmark to execute the function 

 sequentially, one after the other. By running the benchmark with 

 is divided amongst the machine's available threads (distributed amongst multiple goroutines). Having these iterations run concurrently helps to benchmark code that's inherently concurrent, such as sending requests and receiving responses, and deals with mutexes and/or shared resources.

To parallelize benchmarks, wrap the benchmark code in 

func BenchmarkGetPostsHandlerParallel(b *testing.B) {
  b.RunParallel(func(pb *testing.PB) {
    GetPosts = (&JsonPlaceholderMock{}).GetPosts
    r, _ := http.NewRequest("GET", "/posts", nil)
    w := httptest.NewRecorder()
    handler := http.HandlerFunc(PostsResource{}.List)

    b.ReportAllocs()
    b.ResetTimer()

    for pb.Next() {
      handler.ServeHTTP(w, r)
    }
  })
}

Notice that the benchmark values are similar to when we ran the benchmark sequentially with a 

To increase the number of cores (and goroutines) to run this benchmark against, add the 

go test -cpu=1,2,4 -bench=. ./routes -run=^$

Increasing the number of cores increases the number of goroutines running the benchmark iterations, which results in better performance. 

 for a final version of the route handler unit test.

Try writing benchmarks for the other route handlers.

<h1 data-node-type="titlePlaceholder" class="titlePlaceholder">Benchmarking a Go and <code>chi</code> RESTful API</h1><p>The amount of time and effort a developer dedicates towards writing a function depends on the details they choose to focus on: coding conventions, structure, programming style, etc. Suppose a group of developers is presented a high-level prompt to write the same function: given some input, return some output. For example, given a list of numbers, return a sorted list of numbers. The actual implementation of the function is left entirely to the discretion of the developer. A quick, mathematical way to evaluate each developer's implementation of this function, without any additional code, is by its <em>time complexity</em>. Particularly, knowing each implementation's Big-O complexity tells us how it might perform in the worst case scenario, commonly when the size of the input is very large.  </p><p>However, time complexity fails to account for the hardware the function is executed upon, and it does not provide any tangible, quantifiable metrics to base decisions on. Metrics such as <em>operation speed</em> and <em>total execution time</em> assign real numerical values to the performance of a function. By adding <strong>benchmarks</strong>, developers can leverage these metrics to better inform them on how to improve their code. The Go programming language has a <a href="https://golang.org/pkg/testing/#hdr-Benchmarks" target="_blank" rel="noopener noreferrer nofollow">benchmarking utility</a> in its built-in, standard library package <code><a href="https://golang.org/pkg/testing/" target="_blank" rel="noopener noreferrer nofollow">testing</a></code>. To benchmark code in Go, define a function with a name prefixed with <code>Benchmark</code> (followed by a capitalized segment of text) and accepts an argument of struct type <code><a href="https://golang.org/pkg/testing/#B" target="_blank" rel="noopener noreferrer nofollow">B</a></code>, which contains methods and values for determining the number of iterations to run, running multiple benchmarks in parallel, timing execution times, etc. </p><p><em>Example</em>:</p><div style="
          min-height: 232px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-go" data-uniqueid="cm-zTctwmBzAAAv3NNdOjoFw" id="cm-zTctwmBzAAAv3NNdOjoFw">import (
  "testing"
)

func BenchmarkSumFunc(b *testing.B) {
  for i := 0; i &lt; b.N; i++ {
    Sum(100, i)
  }
}</textarea></div><figure class="photo figure-position-inline"><img src="https://www.dl.dropboxusercontent.com/s/dpluuggojgdon0e/Screen%20Shot%202021-02-28%20at%2010.13.46%20PM.png" alt="Benchmarking Simple Functions" contenteditable="false"><div></div></figure><p><em>Note</em>: The structure of a benchmark is similar to the structure of a test. Replace <code>Test</code> with <code>Benchmark</code> and <code>t *testing.T</code> with <code>b *testing.B</code>.</p><p>This benchmark runs the <code>Sum</code> function for <code>b.N</code> iterations. Here, the benchmark runs for one billion iterations, which allows the benchmark function to reliably time and record each execution. Once the benchmark is completed, the results of this benchmark and the CPU of the machine running this benchmark are outputted to the terminal. On average, each iteration ran <code>0.6199</code> ns.</p><p>Below, I'm going to show you...</p><ul><li><p>How to benchmark a route handler of a Go and <code><a href="https://github.com/go-chi/chi" target="_blank" rel="noopener noreferrer nofollow">chi</a></code> RESTful API.</p></li><li><p>How to run benchmarks in parallel.</p></li></ul><h2 class="heading-anchor-text" id="installation-and-setup"><span>Installation and Setup</span><a class="d-inline heading-anchor" title="Direct link to heading" href="#installation-and-setup">#</a></h2><p>Clone a copy of the Go and <code>chi</code> RESTful API from GitHub to you machine:</p><div style="
          min-height: 40px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-sh" data-uniqueid="cm-EL2l10gis83S7YyeTVMod" id="cm-EL2l10gis83S7YyeTVMod">git clone --basic-tests https://github.com/newline-sandbox/go-chi-restful-api.git</textarea></div><p>This RESTful API specifies five endpoints for performing operations on posts:</p><ul><li><p><code>GET /posts</code> - Retrieve a list of posts.</p></li><li><p><code>POST /posts</code> - Creates a post.</p></li><li><p><code>GET /posts/{id}</code> - Retrieve a single post identified by its <code>id</code>. </p></li><li><p><code>PUT /posts/{id}</code> - Update a single post identified by its <code>id</code>. </p></li><li><p><code>DELETE /posts/{id}</code> - Delete a single post identified by its <code>id</code>.</p></li></ul><p>If you would like to learn how to build this RESTful API, then please visit this <a href="https://www.newline.co/@kchan/building-a-simple-restful-api-with-go-and-chi--5912c411" target="_blank" rel="noopener noreferrer nofollow">blog post</a>. In the <code>basic-tests</code> branch, a simple unit test is already provided in the <code>routes/posts_test.go</code> file. Because benchmarks must be placed in <code>_tests.go</code> files, let's place the benchmarks for the <code>posts</code> sub-router in the <code>routes/posts_test.go</code> file.</p><p>Run the following command to install the project's dependencies:</p><div style="
          min-height: 40px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/jsx" data-uniqueid="cm-XO_uNXWOfLIIYopjF1Fpc" id="cm-XO_uNXWOfLIIYopjF1Fpc">$ make install_deps</textarea></div><p><em>Note</em>: If you run into installation issues, then verify that the version of Go running on your machine is <a href="https://golang.org/doc/go1.16" target="_blank" rel="noopener noreferrer nofollow">v1.16</a>.</p><p>Run the following command to execute the unit tests within the <code>routes/posts_test.go</code> file:</p><div style="
          min-height: 40px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/jsx" data-uniqueid="cm-dedlpkEPB-aKh-B7ZNbTK" id="cm-dedlpkEPB-aKh-B7ZNbTK">$ make test</textarea></div><h2 class="heading-anchor-text" id="writing-a-benchmark-for-a-route-handler"><span>Writing a Benchmark for a Route Handler</span><a class="d-inline heading-anchor" title="Direct link to heading" href="#writing-a-benchmark-for-a-route-handler">#</a></h2><p>To get started, open the <code>routes/posts_test.go</code> file. Let's name the benchmark function <code>BenchmarkGetPostsHandler</code>:</p><div style="
          min-height: 88px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-go" data-uniqueid="cm-2pDGU-pMnHJtNHUTkOSRy" id="cm-2pDGU-pMnHJtNHUTkOSRy">func BenchmarkGetPostsHandler(b *testing.B) {
  // ...
}</textarea></div><ol><li><p>Define a sub-test/sub-benchmark with a unique name via <code><a href="https://golang.org/pkg/testing/#hdr-Subtests_and_Sub_benchmarks" target="_blank" rel="noopener noreferrer nofollow">b.Run</a></code>. Using <code>b.Run</code> allows benchmarks to be grouped by the functionality it's testing, and benchmarks can be structured hierarchically with nested benchmarks. Also, in the benchmark's output, Go prints this unique name next to the benchmark's name.<code>go
b.Run("Endpoint: GET /posts", func(b *testing.B) {
  // ...
}</code></p></li><li><p>To avoid sending network requests during benchmarking, mock out the <code>GetPosts</code> package-scoped variable. <code>go
GetPosts = (&amp;JsonPlaceholderMock{}).GetPosts</code></p></li><li><p>Create a new <code>GET</code> request to send to <code>/posts</code> via the <code><a href="https://golang.org/pkg/net/http" target="_blank" rel="noopener noreferrer nofollow">http</a></code> package's <code><a href="https://golang.org/pkg/net/http/#NewRequest" target="_blank" rel="noopener noreferrer nofollow">NewRequest</a></code> method. This request will be passed to the route handler. Since it is not used in subsequent statements in this benchmark, the request error is set to an underscore.<code>go
r, _ := http.NewRequest("GET", "/posts", nil)</code></p></li><li><p>Create a response recorder via <code><a href="https://golang.org/pkg/net/http/httptest/#NewRecorder" target="_blank" rel="noopener noreferrer nofollow">httptest.NewRecorder</a></code> to record the mutations of <code>ResponseWriter</code>, which is passed as an argument to the route handler.<code>go
w := httptest.NewRecorder()</code></p></li><li><p>In the context of this benchmark, the <code>PostsResource{}.List</code> method is an ordinary function. Pass it to the <code><a href="https://golang.org/pkg/net/http" target="_blank" rel="noopener noreferrer nofollow">http</a></code> package's <code><a href="https://golang.org/pkg/net/http/#HandlerFunc" target="_blank" rel="noopener noreferrer nofollow">HandlerFunc</a></code> method to treat it as an HTTP route handler. Since HTTP route handlers have a function signature with the parameters <code>ResponseWriter</code> and <code>Request</code>, and <code>PostsResource{}.List</code> has this same function signature, calling the HTTP route handler will call the <code>PostsResource{}.List</code> method.<code>go
handler := http.HandlerFunc(PostsResource{}.List)</code></p></li><li><p>Turn on malloc statistics via <code><a href="https://golang.org/pkg/testing/#B.ReportAllocs" target="_blank" rel="noopener noreferrer nofollow">b.ReportAllocs</a></code>, which displays an additional two columns in the output of the benchmark. These columns tell how many bytes of memory, on average, were allocated per iteration and how many allocations, on average, were performed per iteration. Having this statement is the same as setting the <code>-benchmem</code> for the <code>go test</code> command.<code>go
b.ReportAllocs()</code></p></li><li><p>Cut out the expensive setup time from the reported execution time via <code><a href="https://golang.org/pkg/testing/#B.ResetTimer" target="_blank" rel="noopener noreferrer nofollow">b.ResetTimer</a></code>.<code>go
b.ResetTimer()</code></p></li><li><p>The <code><a href="https://golang.org/pkg/net/http/#HandlerFunc.ServeHTTP" target="_blank" rel="noopener noreferrer nofollow">handler.ServeHTTP</a></code> method executes the HTTP route handler representation of <code>PostsResource{}.List</code> using the response recorder <code>w</code> and the created request <code>r</code>. Loop over <code>handler.ServeHTTP</code> until the benchmark determines how fast it runs. 
The value of <code>b.N</code> changes until the benchmark stabilizes and knows how long it must run to properly time the function. <code>go
for i := 0; i &lt; b.N; i++ {
  handler.ServeHTTP(w, r)
}</code></p></li></ol><p>Combine the code snippets together:</p><p>(<code>routes/posts_test.go</code>)</p><div style="
          min-height: 424px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-go" data-uniqueid="cm-485GNg75RKJUvd16JgSAQ" id="cm-485GNg75RKJUvd16JgSAQ">// ...

func BenchmarkGetPostsHandler(b *testing.B) {
  b.Run("Endpoint: GET /posts", func(b *testing.B) {
    GetPosts = (&amp;JsonPlaceholderMock{}).GetPosts
    r, _ := http.NewRequest("GET", "/posts", nil)
    w := httptest.NewRecorder()
    handler := http.HandlerFunc(PostsResource{}.List)

    b.ReportAllocs()
    b.ResetTimer()

    for i := 0; i &lt; b.N; i++ {
      handler.ServeHTTP(w, r)
    }
  })
}</textarea></div><p>Run the benchmark:</p><div style="
          min-height: 40px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-sh" data-uniqueid="cm-K2UPbikTD3Gyx6JS-YjsX" id="cm-K2UPbikTD3Gyx6JS-YjsX">$ go test -bench=. ./routes -run=^$</textarea></div><ul><li><p><code>-bench=.</code> - Tells <code>go test</code> to run benchmarks and tests within the project's <code>_test.go</code> files. <code>.</code> is a regular expression that tells <code>go test</code> to match with everything.</p></li><li><p><code>./routes</code> - Tells <code>go test</code> the location of the <code>_test.go</code> files with the benchmarks and tests to run.</p></li><li><p><code>-run=^$</code> - Tells <code>go test</code> to run tests with names that satisfy the regular expression <code>^$</code>. Since none of the tests' names begin with <code>$</code>, <code>go test</code> will not run any tests and will only run benchmarks.  </p></li></ul><figure class="photo figure-position-inline"><img src="https://www.dl.dropboxusercontent.com/s/ozj5qgmsyr03sk2/Screen%20Shot%202021-03-01%20at%203.09.36%20AM.png" alt="Route Handler Benchmark" contenteditable="false"><div></div></figure><p>Here, benchmarks run sequentially.</p><p>The benchmark ran a total of <code>97220</code> iterations with each iteration running, on average, <code>11439</code> ns. This represents the average time it took for each <code>handler.ServeHTTP</code> function (and by extension, <code>PostsResource{}.List</code>) call to complete. Each iteration involved the allocation of, on average, <code>33299</code> bytes of memory. Memory was allocated, on average, <code>8</code> times per iteration.</p><h2 class="heading-anchor-text" id="running-benchmarks-in-parallel"><span>Running Benchmarks in Parallel</span><a class="d-inline heading-anchor" title="Direct link to heading" href="#running-benchmarks-in-parallel">#</a></h2><p>The <code>for</code> loop forces the benchmark to execute the function <code>handler.ServeHTTP</code> sequentially, one after the other. By running the benchmark with <code>b.RunParallel</code>, the total iterations <code>b.N</code> is divided amongst the machine's available threads (distributed amongst multiple goroutines). Having these iterations run concurrently helps to benchmark code that's inherently concurrent, such as sending requests and receiving responses, and deals with mutexes and/or shared resources.</p><p>To parallelize benchmarks, wrap the benchmark code in <code>b.RunParallel</code> and replace the <code>for</code> loop with <code>for pb.Next()</code>:</p><p>(<code>routes/posts_test.go</code>)</p><div style="
          min-height: 376px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-go" data-uniqueid="cm-JeUEmxER2Stw5fPJlOjgw" id="cm-JeUEmxER2Stw5fPJlOjgw">func BenchmarkGetPostsHandlerParallel(b *testing.B) {
  b.RunParallel(func(pb *testing.PB) {
    GetPosts = (&amp;JsonPlaceholderMock{}).GetPosts
    r, _ := http.NewRequest("GET", "/posts", nil)
    w := httptest.NewRecorder()
    handler := http.HandlerFunc(PostsResource{}.List)

    b.ReportAllocs()
    b.ResetTimer()

    for pb.Next() {
      handler.ServeHTTP(w, r)
    }
  })
}</textarea></div><p>Run the benchmark again.</p><figure class="photo figure-position-inline"><img src="https://www.dl.dropboxusercontent.com/s/2nemnoh5m5ag50v/Screen%20Shot%202021-03-01%20at%203.32.05%20AM.png" alt="Parallel Benchmark" contenteditable="false"><div></div></figure><p>Notice that the benchmark values are similar to when we ran the benchmark sequentially with a <code>for</code> loop.</p><p>To increase the number of cores (and goroutines) to run this benchmark against, add the <code>cpu</code> flag:</p><div style="
          min-height: 40px;
          max-width: 100%;
            "><textarea data-iscodemirror="true" class="d-none" data-mode="text/x-sh" data-uniqueid="cm-5-i_mQ7Way_Lwdk4LMsSh" id="cm-5-i_mQ7Way_Lwdk4LMsSh">go test -cpu=1,2,4 -bench=. ./routes -run=^$</textarea></div><figure class="photo figure-position-inline"><img src="https://www.dl.dropboxusercontent.com/s/vbl095eob8dmo6t/Screen%20Shot%202021-03-01%20at%203.39.02%20AM.png" alt="Parallel Benchmark - Multiple Cores" contenteditable="false"><div></div></figure><p>Increasing the number of cores increases the number of goroutines running the benchmark iterations, which results in better performance. </p><h2 class="heading-anchor-text" id="next-steps"><span>Next Steps</span><a class="d-inline heading-anchor" title="Direct link to heading" href="#next-steps">#</a></h2><p>Click <a href="https://github.com/newline-sandbox/go-chi-restful-api/tree/benchmarks" target="_blank" rel="noopener noreferrer nofollow">here</a> for a final version of the route handler unit test.</p><p>Try writing benchmarks for the other route handlers.</p><h2 class="heading-anchor-text" id="sources"><span>Sources</span><a class="d-inline heading-anchor" title="Direct link to heading" href="#sources">#</a></h2><ul><li><p><a href="https://golang.org/pkg/testing/" target="_blank" rel="noopener noreferrer nofollow">Go </a><code><a href="https://golang.org/pkg/testing/" target="_blank" rel="noopener noreferrer nofollow">testing</a></code><a href="https://golang.org/pkg/testing/" target="_blank" rel="noopener noreferrer nofollow"> Package Documentation</a></p></li></ul>

Learn

The newline Guide to Building Your First GraphQL Server with Node and TypeScript

Teach

Amelia Wattenberger

Author of Fullstack D3

Community

Benchmarking a Go and chi RESTful API

Responses (0)

Masterclasses

Tutorials

Fullstack React with TypeScript