Using Polars cuDF after this course

Author

Marie-Hélène Burle

This section is for your future reference. It explains how, after this course, you can use Polars with cuDF on your computer (provided you have suitable hardware) and on the Alliance supercomputers.

For this course however, we have already set everything up for you on a virtual cluster (see next section). So please do not run any of the following during the course.

On your computer

Requirements

To run cuDF on your computer, you need a supported OS and at least one supported GPU. If you don’t, you can use a cloud service or the Alliance supercomputers.

Installation

I suggest using uv to install packages on your computer.

If you have CUDA 12 on your machine, you can simply run:

uv init --bare                        # Initiate project
uv add polars[gpu]

Currently, this is equivalent to installing cudf-polars-cu12.

If you have CUDA 13, install instead with:

uv init --bare
uv add cudf-polars-cu13

The last version of RAPIDS cuDF to support CUDA 11 is cudf-polars-cu11==25.06.

On Alliance clusters

The bad news is that RAPIDS is notoriously hard to install on some systems and that includes the Alliance supercomputers. Our admins are working on trying to add the various libraries to our wheel-house. If they succeed, using them on the clusters will be easy.

Load module

Check available Python versions:

module spider python

Check requirements to load Python 3.14:

module spider python/3.14.2

Double-check that you already have StdEnv/2023:

module list

Load Python 3.14:

module load python/3.14.2

Virtual environment

Create a virtual env:

python -m venv ~/cudf_env

You can name the environment however you like.

Activate it:

source ~/cudf_env/bin/activate

Update pip from our wheel:

python -m pip install --upgrade --no-index pip

Install <rapids-package-of-choice> from our wheel:

python -m pip install --no-index <rapids-package-of-choice>

In the meantime, you can use the RAPIDS packages via an Apptainer image.