Super-resolution with PyTorch

Marie-Hélène Burle

November 24, 2021


Definitions

LR:    low resolution

HR:    high resolution

SR:    super-resolution = reconstruction of HR images from LR images

SISR:   single-image super-resolution = SR using a single input image

History of super-resolution

Can be broken down into 2 main periods:

  • A rather slow history with various interpolation algorithms of increasing complexity before deep neural networks

  • An incredibly fast evolution since the advent of deep learning (DL)

SR history Pre-DL

Pixel-wise interpolation prior to DL

Various methods ranging from simple (e.g. nearest-neighbour, bicubic) to complex (e.g. Gaussian process regression, iterative FIR Wiener filter) algorithms

SR history Pre-DL

Nearest-neighbour interpolation

Simplest method of interpolation

Simply uses the value of the nearest pixel

Bicubic interpolation

Consists of determining the 16 coefficients \(a_{ij}\) in:

\[p(x, y) = \sum_{i=0}^3\sum_{i=0}^3 a\_{ij} x^i y^j\]

SR history with DL

Deep learning has seen a fast evolution marked by the successive emergence of various frameworks and architectures over the past 10 years

Some key network architectures and frameworks:

  • CNN
  • GAN
  • Transformers

These have all been applied to SR

SR using (amongst others):

SRCNN