Teaching programming with LLMs
Reflections on a first experiment
Content from the talk slides for easier browsing.
Rationale
One week is too little to teach Python to beginners:
Using LLMs is the only realistic way to bring them to a place where they can apply knowledge to their research.
Students get frustrated by failures:
They want to make progress on something meaningful. Spending hours debugging error messages about some obscure detail of their code makes them angry and discouraged.
LLMs seemed like the perfect answer for this audience:
This is definitely the future. LLMs can take care of the nitty gritty aspects of the code for them while they focus on the higher level they are interested in.
How I went about it
Course intro
The course started the same way as the traditional course:
- High-level intro to programming
- How to choose a language (FOSS vs proprietary, compiled vs interpreted, etc.)
- High-level intro to Python
- Tools needed for programming (text editor, IDE, etc.)
- Intro to Python shell, IPython, Jupyter, scripts
- Intro to general concepts (packages, syntax, data types, variables, functions, etc.)
Intro to LLMs
First play with text
Web scraping
Reflections on the experiment
Society is changing fast
2024
All I could hear at DHSI was opposition to AI:
- Issues with copyright (valid point!).
- Artists having their work stolen.
- AI being bad and wrong and evil and to be avoided at all cost.
2025
I came prepared to convince them that, despite the (absolutely real) downsides, they have to learn to use LLMs if they don’t want to fall behind and be at a disadvantage.
What I found …
Everybody was way more comfortable than I was using LLMs. Everybody had a subscription. (And everybody was perplexed by my reassuring lecture on the usefulness of LLMs …).
One year is a very long time at the moment.
I was aware that we need to keep up with
- the technology (that’s feasible),
- the regulations (easy for now!).
But I hadn’t anticipated that we also need to keep up with societal change.
What did they think?
Exit surveys had several positive comments about the use of LLMs.
One attendee pointed out however that they don’t need us to use an LLM and that there was no point going to a course if that is what we were doing (good point…).
Will I do it again?
No.
It is too messy in the context of a classroom: everybody goes down their own rabbit hole with their own LLM. LLMs are amazing private tutors but their non-deterministic nature makes them impossible to use in the way I envisaged.
The only way not to have it feel like I am herding squirrels is if I am the only one using an LLM while they passively watch.
What now?
Teaching them to use LLMs is silly: they know!
But now LLMs can teach them everything we can.
So then the question becomes …
Have we become irrelevant?
Maybe not yet.
One attendee had fantastic suggestions in their feedback form. We could:
- Curate a dataset.
- Have them do a more structured and simpler project in this artificially created world.
We need to start teaching differently
We need to go beyond simple teaching if we want to remain relevant.
Organizing a course experience that goes beyond knowledge delivery could be an answer.
(There is more than a hint of unconvinced wishful thinking in all this though …).
LLMs can help us be better instructors. See for instance:
Conclusions
It is probably best to leave LLMs out of the classroom, but we need to realize that all students are using them (probably more than us).
So as not to become irrelevant, we need to adapt and offer fancy interactive curated courses, not just content delivery. LLMs can help us with that since they can create synthetic datasets, come up with exercises, give us ideas, provide feedback, etc.
Epilogue
And then came Gemini 3 and Claude Opus 4.5 …
