A tool to rule them all

Versions, virtual env, and co with pip, pipx, pyenv, virtualenv uv

Author

Marie-Hélène Burle

Content from the webinar slides for easier browsing.

Context

A cluttered toolkit

Age of Rust

uv

  • Universal tool.
  • Really fast.
  • Excellent dependency resolution with PubGrub (you guessed it, also written in Rust).
  • Dependency deduplication.

Warning

Do not use uv on the Alliance clusters. This is for your local computer only.

Following is a recap of a good workflow on the Alliance clusters.

Python versions on Alliance clusters

Use module (uv).

List available Python versions:

module spider python

Check how to load a particular version:

module spider python/3.12.4

Load a particular version:

module load python/3.12.4

Create a Python virtual environment:

python -m venv ~/env

Activate it:

source ~/env/bin/activate

Update pip from wheel:

python -m pip install --upgrade pip --no-index

Use pip with --no-index to use wheels whenever possible:

python -m pip install --no-index jax[cuda12] jax-ai-stack[grain]

Getting started with uv

Install uv

Help

List of commands and options:

uv

List of options:

uv <command> -h    # e.g. uv init -h

Man page:

uv help <command>  # e.g. uv help init

Stuck in a rut

(When you can’t change your workflow.)

Drop-in replacement

You can add uv in front of your usual venv and pip commands.

This actually runs uv (and neither pip nor venv) so you get the speedup, but it keeps everything compatible.

Create a virtual env

uv venv

With specific Python version:

uv venv --python 3.12

By default, the virtual env is called .venv. If you don’t change its name, uv will use it automatically so you don’t need to source it.

Install packages in virtual env

uv pip install jax flax

From GitHub repo:

uv pip install "git+https://github.com/jax-ml/jax"
uv pip install "git+https://github.com/jax-ml/jax@main"
uv pip install "git+https://github.com/jax-ml/jax@766e68c4813a30e29b4fcefaa3253a42d0e197be"

From requirements.txt or pyproject.toml files:

uv pip install -r requirements.txt
uv pip install -r pyproject.toml

All your usual commands work

uv pip uninstall jax
uv pip list
uv pip freeze
…

Python versions

Automatic installation

Missing Python versions are automatically installed when required.

Example:

uv venv --python 3.12

If Python 3.12 is missing, uv will install it during the creation of this virtual env.

Install Python

Python versions can also be installed explicitly:

uv python install 3.12.3
uv python install '>=3.8,<3.10'

Specific implementations (default is cpython):

uv python install pypy
uv python install 'pypy>=3.8,<3.10'

Manage versions

View installed and available versions:

uv python list

Uninstall Python version:

uv python uninstall 3.10

Noe that this is a lot more convenient than pyenv which requires the exact Python version number to uninstall (e.g. pyenv uninstall 3.10.6).

Python projects

Initialize projects

uv init my_project
Initialized project `my-project` at `/home/marie/my_project`

With specific Python version:

uv init --python 3.12 my_project

Customize which files get created:

uv init --no-readme --no-description

Project structure

eza -aT my_project
"my_project": No such file or directory (os error 2)
bat -p my_project/pyproject.toml
[bat error]: 'my_project/pyproject.toml': No such file or directory (os error 2)

Add dependencies

You need to cd into the project, then you can add dependencies:

cd my_project
uv add polars matplotlib

This creates a virtual env called .venv and a uv.lock:

eza -aTL 1
"my_project": No such file or directory (os error 2)

Here again, no need to source the virtual env as long as you use uv.

Project file

Gets populated automatically with dependencies:

bat -p pyproject.toml
[bat error]: 'my_project/pyproject.toml': No such file or directory (os error 2)

List installed dependencies

To list explicitly installed dependencies:

uv tree -d 1
bash: line 1: cd: my_project: No such file or directory
error: No `pyproject.toml` found in current directory or any parent directory

List all dependencies

uv pip list
bash: line 1: cd: my_project: No such file or directory
Using Python 3.12.10 environment at: /home/marie/.local/share/uv/python/cpython-3.12.10-linux-x86_64-gnu
Package Version
------- -------
pip     24.3.1

Manage dependencies

Update all dependencies in lock file and virtual env:

uv sync -U

Remove dependencies:

uv remove matplotlib

Did you say fast?

Python versions pyenv vs uv

pyenv

pyenv install 3.10

This is really taking forever 🥱

uv

uv python install 3.10
Installed Python 3.10.17 in 1.49s

Yes, uv brags about how fast it installs things… but it can!

Packages: pip vs uv pip

pip

Create virtual env:

python -m venv .venv

Activate it:

source .venv/bin/activate

Update pip:

python -m pip install --upgrade pip

Install package:

python -m pip install jax-ai-stack

uv pip

Create virtual env:

uv venv

I am deleting my entire uv cache to make sure that I am not cheating in the comparison. You normally never do that since the cache prevents deduplication (saves space) and makes installations much faster.

rm -rf ~/.cache/uv

Install package:

uv pip install jax-ai-stack

To use the virtual env, I can activate it but I can also access it directly by running commands preceded by uv run.

For instance, I can launch a JupyterLab with access to the project virtual env with:

uv run --with jupyter jupyter lab

or run a script with:

uv run script.py

Convenience

Here is a real world use case: I needed to install a number of packages for a deep learning course with JAX, including Grain which still requires Python 3.12. So I needed a virtual environment with a specific Python version.

Following are the workflows with classic tools vs uv.

pyenv, venv, and pip

Install Python 3.12:

pyenv install 3.12

Create virtual env with Python 3.12 (requires identifying the path):

~/.pyenv/versions/3.12.10/bin/python -m venv .venv

Activate it:

source .venv/bin/activate

Update pip:

python -m pip install --upgrade pip

Install packages:

python -m pip install datasets jax-ai-stack[grain] matplotlib tqdm transformers

uv

uv init --python 3.12 demo

Automatically installs Python 3.12 if missing.

cd demo
uv add datasets jax-ai-stack[grain] matplotlib tqdm transformers

uv advantages

Much simpler.

Much (much!) faster.

Leaves me with a nice pyproject.toml file:

[project]
name = "jxai"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "datasets>=3.5.0",
    "jax-ai-stack[grain]>=2025.2.5",
    "matplotlib>=3.10.1",
    "tqdm>=4.67.1",
    "transformers>=4.50.3",
]

and a uv.lock file that I can put under version control and share for reproducibility.

Tools

pipx replacement

Python tools are packages used for convenience (e.g. linters, formatters) across projects, but not necessary for running your code.

They are commonly installed via your Linux distribution package manager, Homebrew, or pipx.

They can also be installed by uv:

uv tool install ruff

Use tools without installation

Tools can even be used without installation (from a temporary install).

uvx ruff

uvx is an alias for uv tool run.

Resources