Command Line Applications

Clap
0|0|

Many command line applications that we use every day were written long ago when the dominate high level language was C. Ever since there have been improvements made to these fundamental programs as well as attempts to replace them. One niche where Rust fits nicely is in building these command line applications.

One example is ripgrep which can replace grep for many use cases, is faster, and provides a lot of nice extra features. Rust is a big part of the success story of ripgrep.

One feature that is becoming increasingly more important is the ability to support multiple platforms without the need for parallel codebases or ugly hacks. Mac and Linux support have gotten much closer over the years, but Windows is still a much harder target to hit for a wide variety of languages. This is one area that Go has worked quite well at addressing and Rust also shines here as well.

Building command line utilities can greatly simplify your life if you spend a lot of time in the terminal. Sometimes stitching together awk, Perl, and Bash for the third time starts to make you wonder if there is a better way. Before Rust's ecosystem got to the current state it might have made sense to reach for Python or Ruby with their plethora of libraries to build a simple command line tool. But we are now at a point where you can have performance, safety, and a ton of great features by putting together crates out in the wild.

Initial setup#

Let's walk through building a simple command line application for making HTTP requests. This has sufficiently complexity to be interesting, but not too much so that we can focus on the Rust parts of the problem rather than the details of this particular application.

We are going to build something like cURL with a few extra features, so let's create a new project:

Making an MVP#

We are going to walk through quite a bit of code before we get to a runnable version of our application. This is just to get us to an MVP with a few simple features. After that we will expand our feature set with some nice things that curl is missing.

Basic feature set#

We want to support some basic use cases. In all scenarios we want to:

  • Parse the response if it is JSON

  • Print out information about the request including headers

  • Print out the response as nicely as possible

Let's go through a few simple motivating examples:

  • Make a GET request to example.com

Here we see that the default is to make a GET request as no other method is given and no data is being sent.

  • Make a POST request to example.com with a JSON object {"foo":"bar"} as the data:

Here we see the default when data is being sent is to make a POST request. Also, the default is to make a request using JSON rather than a form.

  • Make a PUT request to example.com using HTTPS, set the X-API-TOKEN header to abc123, and upload a file named foo.txt with the form field name info:

We see the -s for --secure option used to turn on HTTPS and -f option used to turn on sending as a form. We want headers to be specified with a colon separating the name from the value and files to be uploaded by using @ between the form field name and the path to the file on disk.

  • Make a form POST request to example.com, use bob as the username for basic authentication, set the query parameter of foo equal to bar, and set the form field option equal to all:

In this last example, before making the request, the user should be prompted for their password which should be used for the basic authentication scheme.

Building the features#

First things first, we need to add dependencies to our manifest:

Whoa that is a lot of dependencies. It is easiest to get most of these out of the way now as working them in after the fact makes things overly complicated. We will get to each of these in detail as we use them, but the quick overview of each is:

  • structopt - command line argument parsing, and much more.

  • heck - converting strings to different cases (e.g. snake_case, camelCase).

  • log - logging facade for outputting verbose information.

  • pretty_env_logger - logging implementation that works with log.

  • serde - (de)serialization of data.

  • serde_json - serde for JSON data.

  • reqwest - HTTP client.

  • rpassword - ask a user for a password without echoing it to the terminal.

We are going to walk through building this application by going file by file as we build out the necessary bits.

The main entry point: main.rs#

As we are building a binary application, we will need some code in main.rs to kick things off. As the functionality here will not be used in a library, and is in fact mostly building on other libraries, we will forgo the advice of creating a separate library that we call into and instead work directly in main.rs.

Let's bring in some imports to get started:

The structopt crate defines a trait StructOpt and a custom derive which allows you to derive that trait for a type you create. These two pieces together create a system for declaratively specifying how your application takes input from the command line.

Underlying structopt is another crate called clap. You can, and many people do, build a command line application directly by interacting with the clap crate. However, structopt abstracts a lot of those details away, as it is mostly boilerplate, and instead allows you to focus on describing your options using the type system.

If this sounds too abstract at the moment, bear with me, as it will all become clear shortly. Nonetheless, we need to import the trait as we will use that functionality in our main function.

We import TitleCase which is also a trait from the heck crate so that we can convert strings into their title cased equivalents. For example, this_string would be converted to ThisString. We could write this logic ourselves, but why bother to worry about all the edge cases when a high quality crate already exists for this purpose.

Finally, the import of the trace macro from the log crate is to enable us to leverage the log crate's nice features to implement a form of verbose logging. Logging in Rust has largely concentrated on the abstraction provided by the log crate.

When you write a library or other tool that wants to generate logs, you use the macros in the log crate to write those messages at various verbosity levels. When you are creating an application where you want to enable seeing these different log messages, you bring in a separate crate (or write some extra code yourself) which provides the concrete implementation of how those macros get turned into output. It is possible to have an implementation that writes everything to a file, or that only turns on some verbosity levels, or that writes to the terminal with pretty colors.

As we are creating a binary, we depend on a concrete implementation of the logging functionality, but for actually generating the messages we just need the macros defined in the log crate.

As part of building this application, we are going to split up the functionality into different modules. We define those modules here that we will soon write so they become part of our application:

As you should recall, this tells the compiler to look for files (or directories) with those names and to insert that code here with the appropriate scoping.

We next use a use statement to bring our to be written error type into scope to make our type signatures easier to write:

One feature we want to support is pretty printing JSON responses. An aspect of pretty printing that can be quite handy is making sure that the keys are sorted. JSON objects do not have a well-defined order so we are free to print them anyway we want. There is an argument that we should faithfully represent the result based on the order of the bytes from the server. However, we are writing our own tool and are therefore free to make decisions such as this. Thus, let's create a type alias that we will be able to use with serde to get an ordered JSON object:

Rust has multiple hash map implementations in the standard library, one in particular is the BTreeMap which stores entires sorted by the key. In this case, we expect our response to have string keys and arbitrary JSON values. This type alias is not doing any work, but we will see how we use it with serde to get the result we are looking for.

We are going to practice a little speculative programming here by writing a main function with code for things that do not yet exist. Given that structure, we then just have to make those things exist and we will be done. Without further ado, our main function:

First, we return a Result, in particular our custom HurlResult with a success type of () meaning we only have a meaningful value to report for errors, and that the Ok case just means everything worked. This shows that the HurlResult type we need to write is generic over the success type as we have seen other result type aliases before.

The first line of our function uses a call to from_args which is defined on the StructOpt trait. Therefore, we need a struct called App in the app module which implements this trait. This does all of the command line argument parsing including exiting if something can't be parsed as well as printing a help message when necessary.

We then have a call to validate which uses the ? operator to exit early if this function fails. This is not part of the argument parsing that StructOpt does. Rather, this is for handling certain constraints on the arguments that StructOpt is unable to enforce. We will see what this is when we get to defining App, but it just shows that sometimes we need workarounds even if a crate does almost everything we need.

There are two further sections. The first uses the log_level method on our app to get a value to setup logging. The pretty_env_logger crate uses environment variables to configure what to print, so we explicitly set the RUST_LOG environment variable based on the value we get. The format is RUST_LOG=binary_name=level where binary_name is not surprisingly the name of your binary and level is one of the five level values that log defines: trace, debug, info, warn, and error. After setting the environment variable, we call pretty_env_logger::init() to actually hook this logging implementation into our use of the macros from the log crate.

The second piece is the heart of our application. We use the cmd (short for command), property on our app to direct what type of request to make. There are two cases, either we got a command which specifies the HTTP verb to use, in that case we use the client module to make the request and then call a handle_response function with the result.

If we did not get a command, i.e. app.cmd matches None, then we are in the default case where we just got a URL. In this case, we make a GET request if we do not have any data arguments, otherwise we make a POST request. We also call a method on the client module to make this request and pipe through to the same handle_response function.

So in either case we end up with a response from our client module that we need to handle. Let's turn to that handler function:

First, the signature. We expect a response as input, which in this case is just the Response type from the reqwest crate, and we return our result type.

The purpose of our tool is to print useful information to standard output and therefore "handling" the response means doing that printing. First, we are going to build up a string based on the metadata about the response. Then we will turn to printing the body.

Our first useful piece of data is the status code of the response as well as the version of HTTP that was used to communicate said response.

The string s is going to be our buffer that we use for building up the output. We start out by using format! to create a String based on the version stored in the response, the status code, and the description of that status. Example results for this string are "HTTP/1.1 200 OK" or "HTTP/1.1 403 Forbidden". We use the Debug output of resp.version() because that is defined to be what we want to show. Sometimes Display is not defined or Debug is what you want to see so don't always think of {:?} as strictly for debugging.

Next up we want to show the headers from the response. We are going to gather them into a vector which we will combine into a string after the fact.

The response type has a headers function which returns a reference to a Header type that gives us access to the headers. We have to explicitly turn it into an iterator by calling the iter so that we can process each key/value pair. We are again using the format! macro to construct a string for each header.

We create a nice to read key by transforming the raw key in the header map into a consistent style. The to_title_case method is available because we imported the TitleCase trait from heck.

We expect the value to have a string representation, but in case that fails we use a quick and dirty "BAD HEADER VALUE" replacement. We do not want to fail the entire response printing if one header value cannot be printed correctly. Luckily we can use the unwrap_or method to easily handle this case.

One special exception is content length. The reqwest crate does not treat content length as a normal header value and instead provides a content_length function on the response type to get this value. Let's use this function to get a content length to add to your list of headers:

Clap
0|0
 

This page is a preview of Fullstack Rust

No discussions yet. Be the first. All notification go to the author.