What changed since 2022

PyTorch moved fast between version 1.13 (late 2022) and the 2.x series of today. Code from that era — in old tutorials, in older repositories, and in the output of coding agents trained on them — often breaks on current versions.

This page is a field guide to the changes that matter in practice, with the old and new idioms side by side.

Pretrained models

The pretrained=True argument is gone. Pretrained weights are now selected through weight enums (and each enum documents its accuracy):

import torchvision.models as models

# Old (≤ 2022)
model = models.resnet50(pretrained=True)

# Now
model = models.resnet50(weights=models.ResNet50_Weights.IMAGENET1K_V2)

The timm library offers an even larger zoo of models behind a single interface:

import timm

model = timm.create_model("resnet50.a1_in1k", pretrained=True)

Transforms

TorchVision transforms have a version 2 API (the v1 API is deprecated). Notable changes: transforms now work on both images and videos, and ToTensor is replaced by ToImage + ToDtype:

import torch
from torchvision.transforms import v2

# Old
transform = transforms.Compose([
    transforms.Resize(256),
    transforms.ToTensor(),
])

# Now
transform = v2.Compose([
    v2.Resize(256),
    v2.ToImage(),
    v2.ToDtype(torch.float32, scale=True),
])

Mixed precision

torch.cuda.amp is deprecated in favor of torch.autocast, and the gradient scaler is unnecessary with bf16 — the default choice on modern GPUs:

# Old
scaler = torch.cuda.amp.GradScaler()
with torch.cuda.amp.autocast():
    output = model(data)
scaler.scale(loss).backward()
scaler.step(optimizer)

# Now (bf16: no scaler needed)
with torch.autocast(device_type="cuda", dtype=torch.bfloat16):
    output = model(data)
loss.backward()
optimizer.step()

Checkpoints

torch.load now defaults to weights_only=True: it refuses to unpickle arbitrary objects (old .pt files could execute code on loading), so loading a full model saved the old way fails loudly. The safe pattern is to save the state dict:

# Save/load the state dict
torch.save(model.state_dict(), "model.pt")
model.load_state_dict(torch.load("model.pt"))

# Or use the safetensors format (the standard of the Hugging Face Hub)
from safetensors.torch import save_model
save_model(model, "model.safetensors")

Devices

The classic .to(device) dance still works, but there are now lighter options:

# Old
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)
data = data.to(device)

# Now: one line at the top, nothing else to remember
torch.set_default_device("cuda")

Or scoped:

with torch.device("cuda"):
    model = nn.Linear(10, 1)    # created directly on the GPU

For device-agnostic code, the torch.accelerator interface abstracts over CUDA, MPS (Apple), and XPU accelerators.

Speed

Three one-liners cover most needs:

model = torch.compile(model)                  # JIT-compile: significant speedups
torch.set_float32_matmul_precision("high")    # allow TF32 on Ampere+ GPUs
optimizer = torch.optim.AdamW(params, lr=1e-3, fused=True)

We come back to these (and to profiling) in the “Faster training” page.

Export

TorchScript (torch.jit.script, torch.jit.trace) is on its way out; its successor is torch.export. ONNX remains the standard when you need to run a model outside PyTorch.

Distributed training

• python -m torch.distributed.launch is long gone: use torchrun.
• Fully sharded data parallelism has a second generation (FSDP2) with a simpler, per-module API: fully_shard. We use it in the scaling part of this course.
• Large or sharded models checkpoint with torch.distributed.checkpoint instead of torch.save.

Gone or moved

• TorchText (NLP) and TorchData are archived — no longer developed. For text, the Hugging Face ecosystem is the answer.
• functorch was merged into PyTorch as torch.func.
• The BetterTransformer fast path became the built-in scaled dot-product attention.

Summary

If you see this (≤ 2022)	Write this instead
pretrained=True	weights=... enums or timm
torchvision.transforms (v1)	torchvision.transforms.v2
torch.cuda.amp.autocast	torch.autocast(...)
GradScaler with bf16	nothing (only fp16 needs a scaler)
torch.load of a full model	save/load state_dict (or safetensors)
torch.jit.script	torch.export
torch.distributed.launch	torchrun
DataPipes	Datasets (HF)

If an agent hands you PyTorch code

Scan for red flags: pretrained=True, torch.cuda.amp, torch.jit, from torchtext, DataPipes, ToTensor(). When something does not run, paste the error back to the agent and ask it to update the code against the current documentation — then verify. This loop is exactly what the “with an assistant” asides in this course demonstrate.