Everything you wanted to know
(and more!)
about PyTorch tensors

Marie-Hélène Burle

January 27, 2022


frontlogo

Acknowledgements

Many drawings in this webinar come from the book:

The section on storage is also highly inspired by it

Using tensors locally

You need to have Python and PyTorch installed

Additionally, you might want to use an IDE such as elpy if you are an Emacs user, JupyterLab, etc.

Note that PyTorch does not yet support Python 3.10 except in some Linux distributions or on systems where a wheel has been built For the time being, you might have to use it with Python 3.9

Using tensors on CC clusters

(In the terminal)

List available wheels and compatible Python versions:

avail_wheels "torch*"

List available Python versions:

module avail python

Get setup:

module load python/3.9.6             # Load a sensible Python version
virtualenv --no-download env         # Create a virtual env
source env/bin/activate              # Activate the virtual env
pip install --no-index --upgrade pip # Update pip
pip install --no-index torch         # Install PyTorch

You can then launch jobs with sbatch or salloc
Leave the virtual env with the command: deactivate

Outline

  • What is a PyTorch tensor?

  • Memory storage

  • Data type (dtype)

  • Basic operations

  • Working with NumPy

  • Linear algebra

  • Harvesting the power of GPUs

  • Distributed operations

Outline

  • What is a PyTorch tensor?

  • Memory storage

  • Data type (dtype)

  • Basic operations

  • Working with NumPy

  • Linear algebra

  • Harvesting the power of GPUs

  • Distributed operations

ANN do not process information directly

Modified from Stevens, E., Antiga, L., & Viehmann, T. (2020). Deep learning with PyTorch. Manning Publications

It needs to be converted to numbers

Modified from Stevens, E., Antiga, L., & Viehmann, T. (2020). Deep learning with PyTorch. Manning Publications

These numbers must be stored in a data structure

PyTorch tensors are Python objects holding multidimensional arrays