Skip to content
James Dawson By James Dawson Principal I
How to setup Python, PyEnv & Poetry on Windows

Once you start doing any significant or longer-term work with a programming language you can often run into problems when different projects require conflicting versions of the language itself or other package dependencies - in this respect, Python is no different.

The classic Python 'Getting Started' guides for people using Windows often suggest a system-wide installation of Python, as this is a quick and simple process. However, this approach soon falls foul of such dependency management issues.

As with other languages (e.g. Node.js, Ruby, GoLang) Python has virtual environment and dependency management tools to help make the developer's life easier. Historically these tools have often grown out of the Linux community and either cannot be used or are tricky to setup on Windows. In this post we'll walk-through the process of configuring a Windows machine with a Python toolchain that will leave you ready to face the challenges of working in Python's diverse eco-system.

TLDR; - If you're just after the script, then check the pre-requisites below and then take a look at this gist - hopefully it helps you get setup in double-quick time or serves as a basis for what you need.

In the rest of this post we will walk-through the process, using the script in the above gist and calling-out some of the more interesting elements.

As a general principle the script avoids any system-wide installations.

Pre-Requisites

The script has the following pre-requisites:

UPDATE: The script has been updated as it inadvertently contained code that would not run under Windows PowerShell v5. (March 2024)

  • Only designed to run on Windows
  • Works with Windows PowerShell and PowerShell Core
  • You need to have a working git installation
  • You'll need to be online

You won't need any version of Python already installed, in fact, if you have any Python versions showing in 'Add/Remove Programs' then you may want to consider uninstalling them, once you've migrated any of your existing projects to use this new toolchain.

If you've ticked all the above, then let's review what the script will do.

How to install a Python Virtual Environment on Windows

The first step is to install a virtual environment. This will give us the ability to have multiple versions of Python installed on our machine and enable individual projects to control which version they use. We'll be using pyenv-win which is a Windows port of pyenv (itself a port of the classic rbenv!) and is exactly what we need to get started.

pyenv is a simple python version management tool. It lets you easily switch between multiple versions of Python. It's simple, unobtrusive, and follows the UNIX tradition of single-purpose tools that do one thing well.

The script installs pyenv-win as follows:

  1. Clones the pyenv-win repository into a pyenv-win folder in your $HOME directory
  2. Adds pyenv-win to your %PATH% environment variable so the command is readily available
  3. Reloads the %PATH% environment variable so you don't have to mess around with restarting your terminal
  4. Installs the specified version of Python, configurable via the mandatory '-PythonVersion' script parameter. You'll be able to install other versions later, but this is will be treated as the initial default.

How to install Python Poetry on Windows

Poetry is a packaging and dependency management tool for Python. Its dependency management features are similar to other such tools in the Python ecosystem (e.g. pip, conda etc.), however the packaging features let you manage dependencies, packaging and publishing to PyPI (or other compatible repositories) using a single configuration file (read Liam's great blog post for more details on this aspect).

Poetry helps you declare, manage and install dependencies of Python projects, ensuring you have the right stack everywhere.

The script installs python-poetry as follows:

  1. Downloads and runs the official python-based installation script, using the default pyenv environment setup above
  2. Adds poetry to your %PATH% environment variable
  3. Reloads the %PATH% environment variable

Using PowerShell to set permanent Environment variables

The environment variable provider built-in to PowerShell is limited to managing environment variables in the context of the current process. Making persistent changes to environment variables requires working with them at a different scope.

This lack of native PowerShell commands is easily side-stepped by using the standard .NET System.Environment class:

# Read system-wide environment variable
$systemPaths = [System.Environment]::GetEnvironmentVariable("FOO", [System.EnvironmentVariableTarget]::Machine)
# Read user-specific environment variable
$userPaths = [System.Environment]::GetEnvironmentVariable("BAR", [System.EnvironmentVariableTarget]::User)

# Set system-wide environment variable
[System.Environment]::SetEnvironmentVariable("FOO", "foobar", [System.EnvironmentVariableTarget]::Machine)
# Set user-specific environment variable
[System.Environment]::SetEnvironmentVariable("BAR", "barfoo", [System.EnvironmentVariableTarget]::User)

How to reload environment variables in a PowerShell session

Updating an environment variable in the above scopes is not dynamic for the current process, meaning that the new value will not be available to the script that made the change. Restarting the terminal is often inconvenient, so instead you can use the following to dynamically reload the environment variable's value.

NOTE: If you use a terminal app that supports multiple tabs, then just closing the tab of the terminal session in question is not always enough to 'see' the updated environment variable. This is because the tabs are often child processes of the terminal application itself, as such they inherit their environment from their parent which only has a snapshot of the environment variables at the point it started. In this situation you need to close all the terminal sessions and restart the whole app.

PS:> $env:PATH = [System.Environment]::GetEnvironmentVariable("PATH", [System.EnvironmentVariableTarget]::User)

Running the helper script

Now you've seen what the script will do, you may want to run it! Open a PowerShell terminal and use the following command to download and execute it:

PS:> Invoke-WebRequest -Uri "https://gist.githubusercontent.com/JamesDawson/4d90e7fcc535c582c617ed553feaf35d/raw" -UseBasicParsing -OutFile $HOME/Downloads/setup-pyenv-poetry-windows.ps1
PS:> & "$HOME/Downloads/setup-pyenv-poetry-windows.ps1"

For help on how to use pyenv-win and python-poetry, please refer to the online documentation:

Using a Python virtual environment with Visual Studio Code / VSCode

Now that your Python toolchain is fully-installed you just need to remember to prepare your terminal session before launching VSCode (or other IDE that doesn't natively support poetry) so that the required virtual environment is available.

Discover your Power BI Maturity Score by taking our FREE 5 minute quiz.

Assuming that you have previously setup poetry in a project and are subsequently returning to work on it, then run the following from the same directory as your pyproject.toml file:

PS:> poetry shell
Spawning shell within C:\Users\...\AppData\Local\pypoetry\Cache\virtualenvs\<project-name>-<some-id>-py<python-version>
PowerShell ...
Copyright (c) Microsoft Corporation.

https://aka.ms/powershell
Type 'help' to get help.

(<project-name>-<some-id>-py<python-version>) PS:> code .

Once you've opened VSCode, the poetry-managed virtual environment should be selectable from the command pallette via commands like:

  • Python: Select Interpreter
  • Notebook: Select Notebook Kernel

The script

For completeness, here's the current version of the script:

[CmdletBinding()]
param (
    [Parameter(Mandatory=$true)]
    [string] $PythonVersion
)

$ErrorActionPreference = "Stop"
$ProgressPreference = "SilentlyContinue"
Set-StrictMode -Version Latest

function _runCommand {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true, Position=0)]
        [string] $Command,
        [switch] $PassThru
    )

    if ($PassThru) {
        $res = Invoke-Expression $Command
    }
    else {
        Invoke-Expression $Command
    }

    if ($LASTEXITCODE -ne 0) {
        Write-Error "'$Command' reported a non-zero status code [$LASTEXITCODE] - check previous output$($PassThru ? ":`n$res" : ".")" 
    }

    if ($PassThru) {
        return $res
    }
}
function _addToUserPath {
    [CmdletBinding()]
    param (
        [Parameter(Mandatory=$true, Position=0)]
        [string] $AppName,
        [Parameter(Mandatory=$true, Position=1)]
        [string[]] $PathsToAdd
    )

    $pathEntries = [System.Environment]::GetEnvironmentVariable("PATH", [System.EnvironmentVariableTarget]::User) -split ";"

    $pathUpdated = $false
    foreach ($pathToAdd in $PathsToAdd) {
        if ($pathToAdd -notin $pathEntries) {
            $pathEntries += $pathToAdd
            $pathUpdated = $true
        }
    }
    if ($pathUpdated) {
        Write-Host "$($AppName): Updating %PATH%..." -f Green
        # Remove any duplicate or blank entries
        $cleanPaths = $pathEntries | Select-Object -Unique | Where-Object { ![string]::IsNullOrEmpty($_) }

        # Update the user-scoped PATH environment variable
        [System.Environment]::SetEnvironmentVariable("PATH", ($cleanPaths -join ";").TrimEnd(";"), [System.EnvironmentVariableTarget]::User)
        
        # Reload PATH in the current session, so we don't need to restart the console
        $env:PATH = [System.Environment]::GetEnvironmentVariable("PATH", [System.EnvironmentVariableTarget]::User)
    }
    else {
        Write-Host "$($AppName): PATH already setup." -f Cyan
    }
}

Write-Host "#################################################" -f White
Write-Host "## Python, pyenv & poetry Windows setup script ##" -f White
Write-Host "#################################################" -f White

# Install pyenv
if (!(Test-Path $HOME/.pyenv)) {
    if ($IsWindows) {
        Write-Host "pyenv: Installing for Windows..." -f Green
        & git clone https://github.com/pyenv-win/pyenv-win.git $HOME/.pyenv
        if ($LASTEXITCODE -ne 0) {
            Write-Error "git reported a non-zero status code [$LASTEXITCODE] - check previous output."
        }
    }
    else {
        Write-Error "This script currently only supports Windows."
    }
}
else {
    Write-Host "pyenv: Already installed." -f Cyan
}

# Add pyenv to PATH
_addToUserPath "pyenv" @(
    "$HOME\.pyenv\pyenv-win\bin"
    "$HOME\.pyenv\pyenv-win\shims"
)

# Install default pyenv python version
$pyenvVersions = _runCommand "pyenv versions" -PassThru | Select-String $PythonVersion
if (!($pyenvVersions)) {
    Write-Host "pyenv: Installing python version $PythonVersion..." -f Green
    _runCommand "pyenv install $PythonVersion"
}
else {
    Write-Host "pyenv: Python version $PythonVersion already installed." -f Cyan
}

# Set pyenv global version
$globalPythonVersion = _runCommand "pyenv global" -PassThru
if ($globalPythonVersion -ne $PythonVersion) {
    Write-Host "pyenv: Setting global python version: $PythonVersion" -f Green
    _runCommand "pyenv global $PythonVersion"
}
else {
    Write-Host "pyenv: Global python version already set: $globalPythonVersion" -f Cyan
}

if (!(Get-Command poetry -ErrorAction Ignore)) {
    $downloadPath = "$HOME/Downloads/install-poetry.py"
    Write-Host "python-poetry: Installing..." -f Green
    Invoke-WebRequest -Uri "https://install.python-poetry.org" `
                      -UseBasicParsing `
                      -OutFile $downloadPath
    try {
        _runCommand "pyenv exec python `"$downloadPath`""
    }
    finally {
        Remove-Item $downloadPath
    }
}
else {
    Write-Host "python-poetry: Already installed." -f Cyan
}

# Add poetry to PATH
_addToUserPath "python-poetry" @("$HOME\AppData\Roaming\Python\Scripts")

# Test poetry is available
_runCommand "poetry --version"

Write-Host "####################" -f Green
Write-Host "## Setup Complete ##" -f Green
Write-Host "####################" -f Green

FAQs

Why can't Visual Studio Code (VSCode) see my python poetry virtual environment? Before launching VSCode from a terminal session, ensure you run 'poetry shell' in the directory containing the project you want to work on.
Can I use pyenv on Windows? Yes you can. There is a windows-specific version called 'pyenv-win' that supports a wide range of Python versions.

James Dawson

Principal I

James Dawson

James is an experienced consultant with a 20+ year history of working across such wide-ranging fields as infrastructure platform design, internet security, application lifecycle management and DevOps consulting - both technical and in a coaching capacity. He enjoys solving problems, particularly those that reduce friction for others or otherwise makes them more effective.