Installing Python 3 and Pyenv on MacOS, Windows, and Linux

We will be using the AWS Lambda runtime for the latest stable version of Python 3 - `python3.7`.

We will be using the AWS Lambda runtime for the latest stable version of Python 3 - python3.7.

AWS supports a few different runtime environments for python, so if we ever need a particular version, we will need to check that it’s supported by lambda first. A full list of supported runtime versions is available on the AWS Documentation page for Lambda Runtimes.

Check for already installed python versions#

If we are using an operating system like Linux or Mac OS - we may already have some version of python installed on our local development machine. This version of python, commonly referred to as the system python, is rarely the latest version that we need for development. We can confirm if we have a system python version installed and see what version it is with the command python --version, as shown below.

The output above indicates that the system version of python installed is much older than 3.7. In other cases, the command above could fail completely, indicting that python is not installed at all. In cases where older are versions present,it is unsafe to update system python directly. Updating it can break certain tools that our operating system needs for essential tasks, including dependency management software. Fortunately, there is a tool called pyenv that will allow us to safely install the latest python version - regardless of whether we have a system python version installed or not.

Install Pyenv#

Pyenv is an open-source tool used for managing multiple python environments. It allows developers to define which version of python to use on a per-project basis. It's similar to the better-known tool that is commonly used in ruby development - rbenv. Prior to pyenv becoming popular, many python developers used virtualenv to manage environment versions. virtualenv is still widely used - but it lacks some crucial features that pyenv provides for standardized installation methods.

We will walk through the most common installation scenarios here, but if you encounter issues, consult the full installation instructions for the project: https://github.com/pyenv/pyenv#installation

Different operating systems have different dependency needs, the latest of which are documented here: https://github.com/pyenv/pyenv/wiki/Common-build-problems

Installing pyenv on Mac OS using homebrew#

Many developers may already have homebrew installed. It's a widely used dependency manager for Mac OS that provides installation packages for many open-source software projects. The commands to install brew are provided below.

  • Install dependencies for brew

  • Install brew

  • Install dependencies pyenv requires

  • If using Mojave or above, use this command to install an additional required dependency:

  • Install pyenv

  • Update the shell to ensure autocomplete for pyenv is functional

Pyenv on Windows#

If using a version of Windows with Windows Subsystem for Linux configured then skip down to instructions for Ubuntu 18.04

Otherwise - we will need to use a pyenv fork, pyenv-win. The instructions underneath the “install by zip” instructions in the pyenv-win wiki provide instructions for pyenv-win installation.

Pyenv on Amazon Linux 2#

The following commands will install the required pyenv dependencies on Amazon Linux 2

Once dependencies are installed, use the pyenv-installer to finish installation.

Pyenv on Ubuntu Linux 18.04 {id:install-pyenv-ubuntu}#

The following commands will install the required pyenv dependencies on Ubuntu Linux 18.04

Once dependencies are installed, use the pyenv-installer to finish installation.

Pyenv Installer (Linux)#

Linux users will find the pyenv-installer helpful for completing the installation, as shown below.

Copy the last 3 lines from the output into your local clipboard from your terminal, and paste those into bottom of the file at ~/.bashrc. Don’t paste from this book directly - the exported PATH variable will be specific to your install.

Steps to paste the output using vim:

  • Open ~/.bashrc with vim

  • vim commands to modify ~/.bashrc

Use pyenv to install python 3.7 (All Operating Systems)#

The command pyenv install --list can be used to fetch a list of all versions of python which are available to be installed with pyenv.

The output from command above can be extremely long though! Since we're using Amazon Linux 2 for examples, we have grep already installed, which will allow us to filter out everything that isn't 3.7.

That's a much more manageable output. In this case, it looks like 3.7.4 is the latest release, so we will install that.

We can install install any number of versions of python side-by-side using the pyenv install command. Each version gets installed into its own local directory, which ensures it will not conflict with our system python, or any other version. Since pyenv has to compile each new version of python it installs, it can take 5-10 minutes to complete an install this way.

So to install python 3.7.4 specifically, we use:

Once the install completes, we can check which versions of python are now available for use with the pyenv versions command.

In the output above, the system version listed is the default 2.7 version we detected when running python --version earlier. The * next to system means that system is currently selected as default python.

Since we want to use the new 3.7 version for development, we are going to need to set it as the global python version so it is used by default in this shell, and future ones we launch. We can change the global at any time by specifying the version of python we wish to use (it must be one we currently have installed).

The pyenv global command for 3.7.4 is shown below.

After that, when we run pyenv versions again, we can verify that 3.7.4 is in use because it has * next to it.

Likewise, when we run python --version, we should see that the installed version matches up.

Install pipenv#

Now that pyenv is configured to use the latest version of python, we will setup pipenv to handle dependency management in python projects. Pipenv provides features that are missing from the older dependency management tool - pip. Using pipenv will ensure we can keep each project's dependency needs installed into virtual environments, and defined in lockfiles.

Let's upgrade pip to the latest version first

We can now install pipenv with the --user flag

Don't use sudo with pip, use --user

It should not be necessary to use sudo with pip. In most cases, simply using the --user flag after pip install will ensure the package gets installed correctly and can be used without resorting to running with sudo.

Install dev modules from pip#

Besides pipenv, there are a few other modules that we'll want to go ahead and install with pip. In general, we'll use pip install --user to install modules that are used across all projects for development purposes - but use pipenv install for project-specific modules that are imported by our lambdas.

The packages we'll want to install across the dev env - awscli, aws-sam-cli, pylint, and cookiecutter can be installed with the command below:

Install project modules with pipenv#

To define and install project-specific dependencies, we will need to create a project folder and navigate to it. Remember to always check your current directory before invoking pipenv - it must always be invoked from within a project's directory to work correctly. We can confirm current terminal location with the command pwd, as shown below

From this new directory, pipenv --python 3.7 to initiate the creation of a Pipfile. This command creates a blank file that pipenv uses to store project dependencies, and the --python 3.7 populates a configuration setting in that Pipfile which locks the environment to python 3.7. Output is shown below:

A file named Pipfile now exists in our project directory. Let's check out its contents.

While it's possible to modify this Pipfile directly, it's better practice to just allow pipenv to make those changes on our behalf. So let's go ahead and install a commonly used library, boto3 for use in this new project. We can install it with the command pipenv install boto3

A note on boto3

The module boto3 is widely used for interacting with the AWS API via python. In fact, it's used so frequently that all python Lambdas come with boto3 already packaged! That said, installing it here will not cause any issues - so it's a safe example to practice with.

If we cat the Pipfile again, we should see a new line under [packages] that shows boto3.

Additionally, we should see a new file in our directory called Pipfile.lock. This is the file that contains the full information about which version of the boto3 module is installed, as well as information about any modules that were installed as dependencies of boto3

Start a new discussion. All notification go to the author.