Modules, packages, and libraries

So far, we have talked about functionality that is available whenever you launch Python. Python however comes with a lot more capabilities, some of it out of the box, some of it after you have installed third-party components.

This session covers the types of additional code that can be loaded in a Python session.

Definitions

Modules are Python files containing reusable code (e.g. functions, constants, utilities).

Packages are collections of modules.

Libraries, technically, are collections of packages, although packages and libraries are often used loosely and interchangeably in Python.

The standard library

Python comes with an extensive standard library. As soon as you launch the program, you can access part of it such as the built-in functions and built-in constants:

Example:

type(3)    # type is a built-in function

int

Most of the standard library however is held in several thematic modules. Each module contains additional functions, constants, and facilities. Before you can use them, you need to load them into your session.

Example: the math module

The math module contains many mathematical functions and constants, including the sqrt function.

This function cannot be accessed directly:

sqrt(9)

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
Cell In[2], line 1
----> 1 sqrt(9)

NameError: name 'sqrt' is not defined

In order to use it, you have two options:

• Load the module, then access the function as a method of the module:

import math
math.sqrt(9)

3.0

You can create an alias for the module:

import math as m
m.sqrt(9)

3.0

This is particularly convenient with modules of longer names.

• Import the function directly:

from math import sqrt
sqrt(9)

3.0

Installing packages on your machine

You can install external packages containing additional functions, constants, datasets, and utilities to extend the capabilities of Python.

The Python Package Index is a public repository of open source packages contributed by users.

Installation of packages can be done via pip or (better and much faster) with the new Python package and project manager uv.

Instead of installing packages system wide or for your user, you can create a semi-isolated Python environment in which you install the packages needed for a particular project. This makes reproducibility and collaboration easier. It also helps handle dependency conflicts. It is a great practice to always use virtual environments.

Some Linux distributions will not let you use pip outside a virtual environment anymore.

uv will automatically install packages in virtual environments.

Traditional workflow with pip

Note that the method with uv in the next section is a better (and much faster) option.

Create a Python virtual environment called env:

python -m venv ~/env

Activate it:

source ~/env/bin/activate

Update pip:

python -m pip install --upgrade pip

Install packages:

python -m pip install <package>

To deactivate a virtual environment, run:

deactivate

Modern workflow with uv

uv is an amazing new package and project manager for Python written in Rust that replaces all the old tools (pip, pyenv, virtualenv, poetry, pipx, etc.). It does everything very well and very fast.

The best approach while using uv is to create projects. In addition to a virtual environment, a project contains a TOML file with a list of dependencies (Python packages), a file setting the Python version for the project, and a lock. This is extremely convenient.

Create a new project:

uv init <project-name>

Install packages in the virtual environment of the project:

uv add <package>

You don’t need to activate the virtual environment as long as you are using uv commands.

If you want to launch Jupyter from a uv project, run:

uv run --with jupyter jupyter lab

If you already have a workflow involving pip that you cannot change and you still would like to use uv, you can use the pip interface. It is not as convenient as the project approach, but it allows you to benefit from uv speed.

For more information on uv, you can have a look at the webinar I gave on this tool in May 2025.

Non-Python dependencies

While uv is fantastic to handle all things Python-related (Python versions, packages, environments, projects, etc.), if you need to install a software stack involving non-Python dependencies on your local machine, you will need to use Mamba, conda, or another package manager that can handle non-Python packages.

Installing packages on the clusters

Don’t use conda or Anaconda on the Alliance clusters. If you really must, do it in a container with Apptainer.

On the Alliance clusters, install packages inside a virtual environment and use Python wheels whenever possible.

You can see whether a wheel is available with avail_wheels <package> or look at the list of available wheels. To install from wheels instead of downloading from PyPI, add the --no-index flag to the install command.

Advantages of wheels:

• compiled for the clusters hardware,
• ensures no missing or conflicting dependencies,
• much faster installation.

The workflow thus looks like:

python -m venv ~/env
source ~/env/bin/activate
python -m pip install --upgrade --no-index pip
python -m pip install --no-index <package>

At this point, uv is not technically supported on Alliance clusters. Some people have had success using it, others have had issues while interacting with the clusters module system.

Installing packages from Jupyter

You can run either of the pip or uv commands from within a Jupyter notebook cell by preceding them with the IPython magic command ! which lets it know that it is a Bash (rather than a Python) command.

Example:

!uv init project