A tool to rule them all

Versions, virtual env, and co with pip, pipx, pyenv, virtualenv uv

noshadow

Marie-Hélène Burle

May 6, 2025


Context

A cluttered toolkit

Age of Rust

uv

  • Universal tool
  • Really fast
  • Excellent dependency resolution with PubGrub (you guessed it, also written in Rust)
  • Dependency deduplication

Warning

Do not use uv on the Alliance clusters. This is for your local computer only

Following is a recap of a good workflow on the Alliance clusters

Python versions on Alliance clusters (uv)

Use module

List available Python versions:

module spider python

Check how to load a particular version:

module spider python/3.12.4

Load a particular version:

module load python/3.12.4

Python packages on Alliance clusters (uv)

Create a Python virtual environment:

python -m venv ~/env

Activate it:

source ~/env/bin/activate

Update pip from wheel:

python -m pip install --upgrade pip --no-index

Use pip with --no-index to use wheels whenever possible:

python -m pip install --no-index jax[cuda12] jax-ai-stack[grain]

Getting started with uv

Install uv

Help

List of commands and options:

uv

List of options:

uv <command> -h    # e.g. uv init -h

Man page:

uv help <command>  # e.g. uv help init

Stuck in a rut

(When you can’t change your workflow)

Drop-in replacement

You can add uv in front of your usual venv and pip commands

This actually runs uv (and neither pip nor venv) so you get the speedup, but it keeps everything compatible

Create a virtual env

uv venv

With specific Python version:

uv venv --python 3.12

By default, the virtual env is called .venv. If you don’t change its name, uv will use it automatically so you don’t need to source it

Install packages in virtual env

uv pip install jax flax

From GitHub repo:

uv pip install "git+https://github.com/jax-ml/jax"
uv pip install "git+https://github.com/jax-ml/jax@main"
uv pip install "git+https://github.com/jax-ml/jax@766e68c4813a30e29b4fcefaa3253a42d0e197be"

From requirements.txt or pyproject.toml files:

uv pip install -r requirements.txt
uv pip install -r pyproject.toml

All your usual commands work

uv pip uninstall jax
uv pip list
uv pip freeze
…

Python versions

Automatic installation

Missing Python versions are automatically installed when required

Example:

uv venv --python 3.12

If Python 3.12 is missing, uv will install it during the creation of this virtual env

Install Python

Python versions can also be installed explicitly:

uv python install 3.12.3
uv python install '>=3.8,<3.10'

Specific implementations (default is cpython):

uv python install pypy
uv python install 'pypy>=3.8,<3.10'

Manage versions

View installed and available versions:

uv python list

Uninstall Python version:

uv python uninstall 3.10

Noe that this is a lot more convenient than pyenv which requires the exact Python version number to uninstall (e.g. pyenv uninstall 3.10.6)

Python projects

Initialize projects

uv init my_project

With specific Python version:

uv init --python 3.12 my_project

Customize which files get created:

uv init --no-readme --no-description

Project structure

eza -aT my_project
"my_project": No such file or directory (os error 2)
bat -p my_project/pyproject.toml
[bat error]: 'my_project/pyproject.toml': No such file or directory (os error 2)

Add dependencies

You need to cd into the project, then you can add dependencies:

cd my_project
uv add polars matplotlib

This creates a virtual env called .venv and a uv.lock:

eza -aTL 1
"my_project": No such file or directory (os error 2)

Here again, no need to source the virtual env as long as you use uv

Project file

Gets populated automatically with dependencies:

bat -p pyproject.toml
[bat error]: 'my_project/pyproject.toml': No such file or directory (os error 2)

List explicitly installed dependencies

uv tree -d 1
bash: line 1: cd: my_project: No such file or directory
error: No `pyproject.toml` found in current directory or any parent directory

List all dependencies

uv pip list
bash: line 1: cd: my_project: No such file or directory
Using Python 3.13.3 environment at: /usr
Package                   Version
------------------------- -------------------------------
absl-py                   2.1.0
adblock                   0.0.0
aiohappyeyeballs          2.6.1
aiohttp                   3.11.18
aiosignal                 1.3.2
aiosmtpd                  1.4.6
annotated-types           0.7.0
anyio                     4.9.0
appdirs                   1.4.4
arandr                    0.1.11
argcomplete               3.5.3
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asciidoc                  10.2.1
asttokens                 3.0.0
async-lru                 2.0.5
atpublic                  5.1
attrs                     24.1.0
autocommand               2.2.2
autokey                   0.96.0
autopage                  0.5.2
babel                     2.15.0
beautifulsoup4            4.13.3
black                     25.1.0
bleach                    6.2.0
borgbackup                1.4.1
bottleneck                1.4.2
btrfsutil                 6.14
build                     1.2.2
cachetools                5.5.0
certifi                   2025.4.26
cffi                      1.17.1
charset-normalizer        3.4.1
click                     8.1.8
cliff                     4.8.0
cmd2                      2.5.11
colorama                  0.4.6
comm                      0.2.2
commonmark                0.9.1
contourpy                 1.3.2
coverage                  7.8.0
cryptography              44.0.2
cycler                    0.12.1
cymem                     2.0.9a3
cython                    3.0.12
dbus-python               1.4.0
debtcollector             3.0.0
debugpy                   1.8.14+0.g8b5b84ae.dirty
decorator                 5.2.1
defusedxml                0.7.1
deprecation               2.1.0
distro                    1.9.0
dnspython                 2.7.0
docopt                    0.6.2
docutils                  0.21.2
dogpile-cache             1.3.3
editables                 0.5
executing                 2.2.0
fastjsonschema            2.21.1
filelock                  3.18.0
flit-core                 3.12.0
fonttools                 4.57.0
fqdn                      1.5.1
frozenlist                1.6.0
fsspec                    2025.3.2
google-auth               2.38.0
google-auth-oauthlib      1.2.1
greenlet                  3.2.1
grpcio                    1.72.0
grpcio-tools              1.72.0
h11                       0.16.0
h5py                      3.13.0
hatch-jupyter-builder     0.9.1
hatchling                 1.27.0
html2text                 2024.2.26
html5lib                  1.1
httpcore                  1.0.9
httplib2                  0.22.0
httpx                     0.28.1
idna                      3.10
importlib-metadata        8.6.1
importlib-resources       6.5.2
inflect                   7.5.0
inflection                0.5.1
iniconfig                 2.1.0
installer                 0.7.0
ipykernel                 6.29.5
ipython                   9.2.0
ipython-pygments-lexers   1.1.1
iso8601                   2.1.0
isodate                   0.7.2
isoduration               20.11.0
jaraco-collections        5.1.0
jaraco-context            6.0.1
jaraco-functools          4.1.0
jaraco-text               4.0.0
jedi                      0.19.2
jinja2                    3.1.5
jmespath                  1.0.1
joblib                    1.4.2
json5                     0.12.0
jsonpatch                 1.33
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2024.10.1
jupyter-cache             1.0.1
jupyter-client            8.6.3
jupyter-core              5.7.2
jupyter-events            0.12.0
jupyter-lsp               2.2.5
jupyter-packaging         0.12.3
jupyter-server            2.15.0
jupyterlab                4.4.1
jupyterlab-lsp            5.1.0
jupyterlab-pygments       0.3.0
jupyterlab-server         2.27.3
jupytext                  1.17.1
keystoneauth1             5.8.0
kiwisolver                1.4.6
libtorrent                2.0.11
lit                       19.1.7.dev0
llfuse                    1.5.0
llvmlite                  0.44.0
markdown                  3.7
markdown-it-py            3.0.0
markupsafe                2.1.5
matplotlib                3.10.1
matplotlib-inline         0.1.7
mdit-py-plugins           0.4.2
mdurl                     0.1.2
meson                     1.8.0
mistune                   3.1.3
more-itertools            10.6.0
msgpack                   1.0.5
multidict                 6.4.3
mypy-extensions           1.0.0
nbclient                  0.10.2
nbconvert                 7.16.6
nbformat                  5.10.4
nest-asyncio              1.6.0
netaddr                   1.3.0
netifaces                 0.11.0
netsnmp-python            1.0a1
networkx                  3.4.2
notebook-shim             0.2.4
numba                     0.61.0+0.g8ec16ceeb.dirty
numexpr                   2.10.2
numpy                     2.2.5
oauth2client              4.1.3
oauthlib                  3.2.2
openstacksdk              4.0.1
opt-einsum                3.4.0
os-service-types          1.7.0
osc-lib                   3.1.0
oslo-config               9.5.0
oslo-i18n                 6.5.0
oslo-serialization        5.6.0
oslo-utils                7.1.0
outcome                   1.3.0.post0
overrides                 7.7.0
packaging                 25.0
pandas                    2.2.3
pandocfilters             1.5.1
parsedatetime             2.6
parso                     0.8.4
pathspec                  0.12.1
pbr                       6.1.1
pexpect                   4.9.0
pickleshare               0.7.5
pillow                    11.2.1
pipx                      1.7.1
platformdirs              4.3.6
playwright                1.52.1.dev0+geec856f5.d20250430
pluggy                    1.5.0
polars                    1.29.0
pooch                     1.8.2
prettytable               3.16.0
prometheus-client         0.21.1
prompt-toolkit            3.0.51
propcache                 0.3.1
protobuf                  6.30.2
psutil                    7.0.0
psycopg2                  2.9.10
ptpython                  3.0.29
ptyprocess                0.7.0
pure-eval                 0.2.3
pyaml                     24.12.0
pyarrow                   19.0.1
pyasn1                    0.6.0
pyasn1-modules            0.4.0
pycairo                   1.28.0
pycparser                 2.22
pydantic                  2.11.4
pydantic-core             2.33.2
pyee                      11.1.0
pyfiglet                  1.0.2
pygments                  2.19.1
pygobject                 3.52.3
pyinotify                 0.9.6
pylatexenc                2.10
pyparsing                 3.2.2
pyperclip                 1.9.0
pyproject-hooks           1.2.0
pyqt5                     5.15.11
pyqt5-sip                 12.17.0
pyqt6                     6.9.0
pyqt6-sip                 13.10.0
pyqt6-webengine           6.9.0
pytest                    8.3.5
pytest-cov                6.1.1
pytest-localserver        0.8.1
python-cinderclient       9.6.0
python-dateutil           2.9.0
python-json-logger        3.3.0
python-keystoneclient     5.6.0
python-magic              0.4.27
python-novaclient         18.9.0
python-openstackclient    7.4.0
python-slugify            8.0.4
python-xlib               0.33
pytz                      2025.2
pyyaml                    6.0.2
pyzmq                     26.2.0
qscintilla                2.14.1
qutebrowser               3.5.0
ranger-fm                 1.9.4
referencing               0.35.1
requests                  2.32.3
requests-file             2.1.0
requests-oauthlib         1.3.1
requestsexceptions        1.4.0
rfc3339-validator         0.1.4
rfc3986                   2.0.0
rfc3987                   1.3.8
rich                      14.0.0
rich-cli                  1.8.0
rich-rst                  1.3.1
rpds-py                   0.22.3
rsa                       4.9
ruamel-yaml               0.18.10
ruamel-yaml-clib          0.2.12
scikit-learn              1.6.1
scipy                     1.15.2
send2trash                1.8.3
setuptools                80.0.0
six                       1.17.0
sniffio                   1.3.1
sortedcontainers          2.4.0
soupsieve                 2.6
sqlalchemy                2.0.40
stack-data                0.6.3
standard-cgi              3.13.0
stevedore                 5.4.0
tabulate                  0.9.0
tbb                       0.2
tensorboard               2.19.0
tensorboard-data-server   0.8.0a0
tensorboard-plugin-wit    1.8.1
termdown                  1.18.0
text-unidecode            1.3
textual                   3.2.0
threadpoolctl             3.5.0
tinycss2                  1.4.0
tldextract                5.3.0
toml                      0.10.2
tomlkit                   0.13.2
toolz                     1.0.0
tornado                   6.4.2
tqdm                      4.67.1
traitlets                 5.14.3
trash-cli                 0.24.5.26
trio                      0.30.0
trove-classifiers         2025.4.28.22
typeguard                 4.4.2
types-python-dateutil     2.9.0.20241206
typing-extensions         4.13.2
typing-inspection         0.4.0
ueberzug                  18.3.1
uri-template              1.3.0
urllib3                   2.4.0
userpath                  1.9.2
wcwidth                   0.2.13
webcolors                 24.11.1
webencodings              0.5.1
websocket-client          1.8.0
werkzeug                  3.1.3
wheel                     0.45.1
wrapt                     1.16.0
xlsx2csv                  0.8.1
yarl                      1.20.0
yt-dlp                    2025.4.30
zipp                      3.21.0

Manage dependencies

Update all dependencies in lock file and virtual env:

uv sync -U

Remove dependencies:

uv remove matplotlib

Did you say fast?

Python versions pyenv vs uv

pyenv

pyenv install 3.10

uv

uv python install 3.10
Installed Python 3.10.17 in 1.49s

Yes, uv brags about how fast it installs things… but it can!

Packages: pip vs uv pip

pip

Create virtual env:

python -m venv .venv

Activate it:

source .venv/bin/activate

Update pip:

python -m pip install --upgrade pip

Install package:

python -m pip install jax-ai-stack

Packages: pip vs uv pip

uv pip

Create virtual env:

uv venv

I am deleting my entire uv cache to make sure that I am not cheating in the comparison. You normally never do that since the cache prevents deduplication (saves space) and makes installations much faster

rm -rf ~/.cache/uv

Install package:

uv pip install jax-ai-stack

Packages: pip vs uv pip

uv pip

To use the virtual env, I can activate it but I can also access it directly by running commands preceded by uv run

For instance, I can launch a JupyterLab with access to the project virtual env with:

uv run --with jupyter jupyter lab

or run a script with:

uv run script.py

Convenience

Use case: virtual env with specific Python version

I needed to install a number of packages for a deep learning course with JAX, including Grain which still requires Python 3.12

Following are the workflows with classic tools vs uv

pyenv, venv, and pip

Install Python 3.12:

pyenv install 3.12

Create virtual env with Python 3.12 (requires identifying the path):

~/.pyenv/versions/3.12.10/bin/python -m venv .venv

Activate it:

source .venv/bin/activate

Update pip:

python -m pip install --upgrade pip

Install packages:

python -m pip install datasets jax-ai-stack[grain] matplotlib tqdm transformers

uv

uv init --python 3.12 demo

Automatically installs Python 3.12 if missing

cd demo
uv add datasets jax-ai-stack[grain] matplotlib tqdm transformers

uv advantages

Much simpler

Much (much!) faster

Leaves me with a nice pyproject.toml file:

[project]
name = "jxai"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = [
    "datasets>=3.5.0",
    "jax-ai-stack[grain]>=2025.2.5",
    "matplotlib>=3.10.1",
    "tqdm>=4.67.1",
    "transformers>=4.50.3",
]

and a uv.lock file that I can put under version control and share for reproducibility

Tools

pipx replacement

Python tools are packages used for convenience (e.g. linters, formatters) across projects, but not necessary for running your code

They are commonly installed via your Linux distribution package manager, Homebrew, or pipx

They can also be installed by uv:

uv tool install ruff

Use tools without installation

Tools can even be used without installation (from a temporary install)

uvx ruff

uvx is an alias for uv tool run

Resources

GitHub repo

Website

 Back to webinar page

A tool to rule them all Versions, virtual env, and co with pip, pipx, pyenv, virtualenv uv Marie-Hélène Burle May 6, 2025

  1. Slides

  2. Tools

  3. Close
  • A tool to rule them all
  • Context
  • A cluttered toolkit
  • Age of Rust
  • uv
  • Warning
  • Python versions on Alliance clusters (uv)
  • Python packages on Alliance clusters (uv)
  • Getting started with uv
  • Install uv
  • Help
  • Stuck in a rut
  • Drop-in replacement
  • Create a virtual env
  • Install packages in virtual env
  • All your usual commands work
  • Python versions
  • Automatic installation
  • Install Python
  • Manage versions
  • Python projects
  • Initialize projects
  • Project structure
  • Add dependencies
  • Project file
  • List explicitly installed dependencies
  • List all dependencies
  • Manage dependencies
  • Did you say fast?
  • Python versions pyenv vs uv
  • Packages: pip vs uv pip
  • Packages: pip vs uv pip
  • Packages: pip vs uv pip
  • Convenience
  • Use case: virtual env with specific Python version
  • pyenv, venv, and pip
  • uv
  • uv advantages
  • Tools
  • pipx replacement
  • Use tools without installation
  • Resources
  • f Fullscreen
  • s Speaker View
  • o Slide Overview
  • e PDF Export Mode
  • r Scroll View Mode
  • ? Keyboard Help