Modules, packages, and libraries

Author

Marie-Hélène Burle

Definitions

“Modules” are Python files containing reusable code (e.g. functions, constants, utilities).

“Packages” are collections of modules.

“Libraries”, technically, are collections of packages, although “packages” and “libraries” are often used loosely and interchangeably in Python.

Installing external packages

You can install external packages containing additional functions, constants, and utilities to extend the capabilities of Python.

The Python Package Index is a public repository of open source packages contributed by users.

Installation of packages can be done via pip:

pip install --no-index --upgrade pip
python -m pip install <package>

On your local machine, particularly if you are on Windows and want to install a complex software stack, conda can makes things easy by installing from the Anaconda Distribution. This is however never what you want to do when using the Alliance clusters.

On the clusters, you always want to:

  1. create a virtual environment,
  2. install packages in it with pip.

Virtual environments

Instead of installing packages system wide or for your user, you can create a semi-isolated Python environment in which you install the packages needed for a particular project. This makes reproducibility and collaboration easier. It also helps handle dependency conflicts.

Installing packages on the clusters

On the Alliance clusters, you should install packages inside a virtual environment.