Running Python

Author

Marie-Hélène Burle

This section covers some of the many ways to run Python either on your machine or on the Alliance clusters.

On your machine

Python script

You can write your Python code in a text file with a .py extension and run the script in your terminal with:

python <script-name>.py

This will execute the code non-interactively.

Interactive Python

One key reason why Python is so popular is that it is an interpreted language: you can use it in an interactive session that makes prototyping code very friendly.

Python shell

The simplest (and driest) way to run Python interactively is to use the Python shell. If you launch Python by double-clicking on the Python executable or running the command python in a terminal, you end up in the Python shell with its typical >>> prompt.

IPython

IPython is an improved shell with better performance and more functionality (e.g. colour-coding, magic commands).

You can install IPython and launch it instead of launching Python by running ipython in your terminal.

Now that everybody is using Jupyter (see below), using the IPython shell directly has fallen out of fashion, but it is actually my favourite method to run Python. It is also the most efficient way to run Python interactively.

Jupyter

The IPython shell was integrated into a fancy interface, the Jupyter notebook. This later lead to a fully fledged IDE (integrated development environment) called JupyterLab which contains notebooks, a command line, a file explorer, and other functionality.

You can install JupyterLab, launch it by running jupyter lab in a terminal and the IDE will open in your browser.

Even though JupyterLab runs in your browser, it does not use internet: it is all run locally.

Other IDEs

Jupyter has probably become the most popular IDE, but it is possible to run Python in other IDE such as Emacs.

Quarto

The very popular RMarkdown developed by Posit (formerly RStudio Inc) lead to a new and more powerful tool called Quarto. Quarto runs code blocks of R, Julia, and Python in markdown documents which can be rendered into websites, pdfs, presentations, and more. This website is built with Quarto.

Under the hood, Quarto runs Jupyter notebooks, so it is in fact IPython running in Jupyter that executes the Python code in Quarto.

On Alliance clusters

Python script

You can SSH into an Alliance cluster, load the Python module with the Python version of your choice (use module spider python to find the module, then module load to load it), write a Python script, an sbatch script and run your code with a batch job as you saw in our HPC course.

Interactive Python

Python shell

Similarly, if you SSH to a cluster, load the Python module of your choice, then launch an interactive job with salloc, you can run the Python shell.

IPython

To use IPython, load the IPython module of your choice (module spider ipython to find it, module load to load it), launch an interactive job with salloc, and finally launch the IPython shell by running ipython in the terminal.

Jupyter

To use JupyterLab on a cluster, you use what is called a JupyterHub: a set of tools that spawn and manage multiple instances of JupyterLab servers. Under the hood, they manage an interactive job used by your JupyterLab server.

Let’s try it on our training cluster:

  • go to https://jupyter.school.c3.ca,
  • sign in with the username and password for our summer school,
  • leave OTP blank,
  • set the server options according to the image below:

These are the only values that you should edit:
Change the time to 3.0

  • press start.

Note that, unlike other JupyterHubs you might have used (e.g. Syzygy), this JupyterHub is not permanent and will be destroyed at the end of the summer school.

If you don’t need all the time you asked for after all, it is a great thing to log out (the resources you are using on this cluster are shared amongst many people and when resources are allocated to you, they aren’t available to other people. So it is a good thing not to ask for unnecessary resources and have them sit idle when others could be using them).

To log out, click on “File” in the top menu and select “Log out” at the very bottom.

If you would like to make a change to the information you entered on the server option page after you have pressed “start”, log out in the same way, log back in, edit the server options, and press start again.

Start a Python notebook

To start a Jupyter notebook with the Python kernel, click on the button “Python 3” in the “Notebook” section (top row of buttons).