How to setup Python, PyEnv & Poetry on Windows

Once you start doing any significant or longer-term work with a programming language you can often run into problems when different projects require conflicting versions of the language itself or other package dependencies - in this respect, Python is no different.

The classic Python 'Getting Started' guides for people using Windows often suggest a system-wide installation of Python, as this is a quick and simple process. However, this approach soon falls foul of such dependency management issues.

As with other languages (e.g. Node.js, Ruby, GoLang) Python has virtual environment and dependency management tools to help make the developer's life easier. Historically these tools have often grown out of the Linux community and either cannot be used or are tricky to setup on Windows. In this post we'll walk-through the process of configuring a Windows machine with a Python toolchain that will leave you ready to face the challenges of working in Python's diverse eco-system.

In the rest of this post we will walk-through the process, using the script in the above gist and calling-out some of the more interesting elements.

Pre-Requisites

The script has the following pre-requisites:

• Only designed to run on Windows
• Works with Windows PowerShell and PowerShell Core
• You need to have a working git installation
• You'll need to be online

If you've ticked all the above, then let's review what the script will do.

How to install a Python Virtual Environment on Windows

The first step is to install a virtual environment. This will give us the ability to have multiple versions of Python installed on our machine and enable individual projects to control which version they use. We'll be using pyenv-win which is a Windows port of pyenv (itself a port of the classic rbenv!) and is exactly what we need to get started.

The script installs pyenv-win as follows:

1. Clones the pyenv-win repository into a pyenv-win folder in your $HOME directory
2. Adds pyenv-win to your %PATH% environment variable so the command is readily available
3. Reloads the %PATH% environment variable so you don't have to mess around with restarting your terminal
4. Installs the specified version of Python, configurable via the mandatory '-PythonVersion' script parameter. You'll be able to install other versions later, but this is will be treated as the initial default.

How to install Python Poetry on Windows

Poetry is a packaging and dependency management tool for Python. Its dependency management features are similar to other such tools in the Python ecosystem (e.g. pip, conda etc.), however the packaging features let you manage dependencies, packaging and publishing to PyPI (or other compatible repositories) using a single configuration file (read Liam's great blog post for more details on this aspect).

The script installs python-poetry as follows:

1. Downloads and runs the official python-based installation script, using the default pyenv environment setup above
2. Adds poetry to your %PATH% environment variable
3. Reloads the %PATH% environment variable

Using PowerShell to set permanent Environment variables

The environment variable provider built-in to PowerShell is limited to managing environment variables in the context of the current process. Making persistent changes to environment variables requires working with them at a different scope.

This lack of native PowerShell commands is easily side-stepped by using the standard .NET System.Environment class:

# Read system-wide environment variable
$systemPaths = [System.Environment]::GetEnvironmentVariable("FOO", [System.EnvironmentVariableTarget]::Machine)
# Read user-specific environment variable
$userPaths = [System.Environment]::GetEnvironmentVariable("BAR", [System.EnvironmentVariableTarget]::User)

# Set system-wide environment variable
[System.Environment]::SetEnvironmentVariable("FOO", "foobar", [System.EnvironmentVariableTarget]::Machine)
# Set user-specific environment variable
[System.Environment]::SetEnvironmentVariable("BAR", "barfoo", [System.EnvironmentVariableTarget]::User)

How to reload environment variables in a PowerShell session

Updating an environment variable in the above scopes is not dynamic for the current process, meaning that the new value will not be available to the script that made the change. Restarting the terminal is often inconvenient, so instead you can use the following to dynamically reload the environment variable's value.

PS:> $env:PATH = [System.Environment]::GetEnvironmentVariable("PATH", [System.EnvironmentVariableTarget]::User)

Running the helper script

Now you've seen what the script will do, you may want to run it! Open a PowerShell terminal and use the following command to download and execute it:

PS:> Invoke-WebRequest -Uri "https://gist.githubusercontent.com/JamesDawson/4d90e7fcc535c582c617ed553feaf35d/raw" -UseBasicParsing -OutFile $HOME/Downloads/setup-pyenv-poetry-windows.ps1
PS:> & "$HOME/Downloads/setup-pyenv-poetry-windows.ps1"

For help on how to use pyenv-win and python-poetry, please refer to the online documentation:

• pyenv-win - https://github.com/pyenv-win/pyenv-win#usage
• python-poetry - https://python-poetry.org/

Using a Python virtual environment with Visual Studio Code / VSCode

Now that your Python toolchain is fully-installed you just need to remember to prepare your terminal session before launching VSCode (or other IDE that doesn't natively support poetry) so that the required virtual environment is available.

Assuming that you have previously setup poetry in a project and are subsequently returning to work on it, then run the following from the same directory as your pyproject.toml file:

PS:> poetry shell
Spawning shell within C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\<project-name>-<some-id>-py<python-version>
PowerShell ...
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

(<project-name>-<some-id>-py<python-version>) PS:> code .

Once you've opened VSCode, the poetry-managed virtual environment should be selectable from the command pallette via commands like:

• Python: Select Interpreter
• Notebook: Select Notebook Kernel

The script

For completeness, here's the current version of the script:

[CmdletBinding()]
param (
    [Parameter(Mandatory=$true)]
    [string] $PythonVersion
)

$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
Set-StrictMode -Version Latest

function _runCommand {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true, Position=0)]
        [string] $Command,
        [switch] $PassThru
    )

    if ($PassThru) {
        $res = Invoke-Expression $Command
    }
    else {
        Invoke-Expression $Command
    }

    if ($LASTEXITCODE -ne 0) {
        Write-Error "'$Command' reported a non-zero status code [$LASTEXITCODE] - check previous output$($PassThru ? ":`n$res" : ".")" 
    }

    if ($PassThru) {
        return $res
    }
}
function _addToUserPath {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true, Position=0)]
        [string] $AppName,
        [Parameter(Mandatory=$true, Position=1)]
        [string[]] $PathsToAdd
    )

    $pathEntries = [System.Environment]::GetEnvironmentVariable("PATH", [System.EnvironmentVariableTarget]::User) -split ";"

    $pathUpdated = $false
    foreach ($pathToAdd in $PathsToAdd) {
        if ($pathToAdd -notin $pathEntries) {
            $pathEntries += $pathToAdd
            $pathUpdated = $true
        }
    }
    if ($pathUpdated) {
        Write-Host "$($AppName): Updating %PATH%..." -f Green
        # Remove any duplicate or blank entries
        $cleanPaths = $pathEntries | Select-Object -Unique | Where-Object { ![string]::IsNullOrEmpty($_) }

        # Update the user-scoped PATH environment variable
        [System.Environment]::SetEnvironmentVariable("PATH", ($cleanPaths -join ";").TrimEnd(";"), [System.EnvironmentVariableTarget]::User)
        
        # Reload PATH in the current session, so we don't need to restart the console
        $env:PATH = [System.Environment]::GetEnvironmentVariable("PATH", [System.EnvironmentVariableTarget]::User)
    }
    else {
        Write-Host "$($AppName): PATH already setup." -f Cyan
    }
}

Write-Host "#################################################" -f White
Write-Host "## Python, pyenv & poetry Windows setup script ##" -f White
Write-Host "#################################################" -f White

# Install pyenv
if (!(Test-Path $HOME/.pyenv)) {
    if ($IsWindows) {
        Write-Host "pyenv: Installing for Windows..." -f Green
        & git clone https://github.com/pyenv-win/pyenv-win.git $HOME/.pyenv
        if ($LASTEXITCODE -ne 0) {
            Write-Error "git reported a non-zero status code [$LASTEXITCODE] - check previous output."
        }
    }
    else {
        Write-Error "This script currently only supports Windows."
    }
}
else {
    Write-Host "pyenv: Already installed." -f Cyan
}

# Add pyenv to PATH
_addToUserPath "pyenv" @(
    "$HOME\.pyenv\pyenv-win\bin"
    "$HOME\.pyenv\pyenv-win\shims"
)

# Install default pyenv python version
$pyenvVersions = _runCommand "pyenv versions" -PassThru | Select-String $PythonVersion
if (!($pyenvVersions)) {
    Write-Host "pyenv: Installing python version $PythonVersion..." -f Green
    _runCommand "pyenv install $PythonVersion"
}
else {
    Write-Host "pyenv: Python version $PythonVersion already installed." -f Cyan
}

# Set pyenv global version
$globalPythonVersion = _runCommand "pyenv global" -PassThru
if ($globalPythonVersion -ne $PythonVersion) {
    Write-Host "pyenv: Setting global python version: $PythonVersion" -f Green
    _runCommand "pyenv global $PythonVersion"
}
else {
    Write-Host "pyenv: Global python version already set: $globalPythonVersion" -f Cyan
}

if (!(Get-Command poetry -ErrorAction Ignore)) {
    $downloadPath = "$HOME/Downloads/install-poetry.py"
    Write-Host "python-poetry: Installing..." -f Green
    Invoke-WebRequest -Uri "https://install.python-poetry.org" `
                      -UseBasicParsing `
                      -OutFile $downloadPath
    try {
        _runCommand "pyenv exec python `"$downloadPath`""
    }
    finally {
        Remove-Item $downloadPath
    }
}
else {
    Write-Host "python-poetry: Already installed." -f Cyan
}

# Add poetry to PATH
_addToUserPath "python-poetry" @("$HOME\AppData\Roaming\Python\Scripts")

# Test poetry is available
_runCommand "poetry --version"

Write-Host "####################" -f Green
Write-Host "## Setup Complete ##" -f Green
Write-Host "####################" -f Green