Skip to content

Simplify your Streamlit Python Development Experience with Dev Containers

Carmel Eve Carmel Eve

Show & Tell

Carmel Eve returned to endjin after a 3 year break to travel the world. She left a .NET Developer, and came back to discover a plethora of Python based data & analytics projects in-flight. Carmel has been recording her learning curve adapting to this new technology stack.

In this video, she guides you through setting up and utilizing dev containers to create a controlled, repeatable, and isolated Python development environment. In this video you will learn how to install WSL and Docker Desktop on Windows, configure VS Code with Docker and DevContainers extensions, and initialize your Python project with Poetry.

By following these steps, you can eliminate the hassle of managing dependencies and version conflicts across different projects. Perfect for any developer looking to streamline their Python development workflow.

  • 00:00 Introduction to Local Development Containers
  • 00:12 Challenges with Python Environment Management
  • 00:41 Introduction to Dev Containers
  • 01:28 Setting Up a Local Python Container
  • 01:32 Installing WSL and Docker
  • 02:55 Configuring VS Code for Dev Containers
  • 04:31 Using Poetry for Dependency Management
  • 06:51 Finalizing the Dev Container Setup
  • 07:47 Conclusion and Next Steps

Transcript

In this video, I'll be running through how local development containers can help with the managing of Python environments.

Since returning to endjin, I have entered a whole new world of Python. And on this re entry, I have quickly become frustrated with just how much effort is involved in managing a new Python environment. You need to install so many different packages, and many of these packages are configured to a specific Python version. Managing all this is an absolute nightmare between different environments and different projects. Plus, once you do get it all set up, anyone else opens A project needs to go through exactly the same painful process before they too can start work.

Luckily, there is a solution dev containers. So what is a dev container? Dev containers are docker containers that are specifically configured to provide a fully featured development environment. They're the technology behind GitHub code spaces and provide a way to have a repeatable, controlled, and confined local environment. Not only is the repeatability of the environment incredibly important when using Python, as it means you can easily save and reproduce all the messy package and language version configuration, but the confinement of the environment is also important because it means you can install additional packages, some of which include additional C code that needs to be run as part of the install, without polluting your machine unnecessarily.

So how do we set up a local Python container? First, assuming you are running on Windows, you will need to install WSL. The Windows Subsystem for Linux. WSL is a program that allows you to run a Linux kernel inside of your Windows operating system. It is designed to allow a seamless experience for those needing to run both Windows and Linux concurrently. You can install WSL from the command line using wsl install when running in administrator mode. You will also then need to install Docker Desktop. Docker is used to build, package, and deploy applications in the form of containers. Our dev containers will use docker to manage the environment. Docker Desktop provides an integration with WSL, so it seems as though docker is running inside of Windows.

You can install this from the command line using winget install Docker.DockerDesktop or from the docker website. Docker should work reasonably smoothly out of the box, but it also does have some well documented prerequisites that are worth having a read through on install. Combining these, we have a way to run a non Windows based container on a Windows environment. Microsoft Mechanics without needing to do any of the management or integration ourselves. So now, let's open a new project in VS Code, where we will be building our Python application as the series goes on, and we'll create an initial Python file. But before we start building any sort of application, let's set up our environment.

You will need to install both Docker and the DevContainers extensions in VS Code. If you now click on this icon in the bottom left, likely after the mandatory VS Code restart, it opens the devcontainer menu. If we select reopen in devcontainer, we are then given the option to create a new Python devcontainer configuration file. But this example will just use the defaults and won't add any additional configuration.

This will automatically create a dev container JSON file, which VS Code will then use to build the container that you open the repository inside because we chose to create a Python based dev container. The container will be based on the Microsoft Python container image and we can then configure it by further editing this file vs. Code will then reopen our repository in the newly built dev container. If we want to reopen our machine, we can reopen the container menu and select reopen folder locally.

Now that we are running inside of our container, we can start configuring our environment. First, let's run a pip install for Poetry. Poetry is a tool which helps us manage dependencies and keep them consistent within a Python project. Another thing that can be challenging. It does this by creating a virtual environment in which dependencies are installed. It can also do other things such as build projects for distribution, but I won't get into that here. Once the poetry install has finished, we should be able to use the poetry command to add any packages we need, and all of the dependencies will be managed for us. So, we can now initialize the environment by using the command poetry init.

This will then guide us through the setup of a project. Here I will just use the defaults, and I will add the packages manually later, so we can leave these empty for now. This will create an empty poetry file where our dependencies will be defined. It will also create the local poetry Python virtual environment, where these packages will be installed.

So, let's add a few packages that we know that we'll need. First, we'll add pandas for data processing. And I'm also going to add streamlet, which is a Python package used for surfacing graphs and visuals, that we could use to run a future application. Now, in order to start using these packages, we will need to ensure that the terminal, and any Python files, are pointed at the interpreter in Poetry's virtual environment. If we don't do this, we won't have access to the packages we have installed, and we can see this if we try to run a streamlet command in the current terminal.

And, inside the Python file, we don't currently have access to pandas, despite having installed it. Switch to the correct interpreter, make sure you are inside the Python file, and click here in the bottom right. You should see the environment listed here. If you don't, try restarting VS Code.

Once we have done this, we can reopen a terminal in that virtual environment, and start to use Streamlit commands within that terminal. We can also see that our Python file is now recognising it has accessed the pandas package. Finally, we can add some devcontainer. json file so that when the devcontainer is built, the environment is automatically created and the necessary packages are installed. If we add the pip install poetry command and the poetry install command,then, when the devcontainer starts up, it will check for the poetry lock file and install any dependencies. And, we can see that if we rebuild the container, then,

Everything just works.

So, we are now ready to get started on developing our application. See future videos on how we might build an application using Streamlit inside this dev container.