Thirty-three lessons across six modules that build a GPT from an empty file using only Python and NumPy, derived from Andrej Karpathy's MicroGPT, published as a static course site with every equation rendered and every diagram drawn in code.
Problem
Most transformer explainers stop at the diagram. They draw attention beautifully, then hand you PyTorch and say the rest is an exercise. The consequence is a large population of engineers who can discuss multi-head attention in a meeting and cannot write the backward pass.
Karpathy's MicroGPT attacks exactly that gap, in roughly 300 lines. The problem with it as a teaching artifact is that it is a proof of elegance rather than a curriculum - the code is minimal, which means every line is load-bearing and none of them are explained.
What it does
The course takes the same destination and adds the road. Each lesson is a single commit-sized step: build the tokeniser, write the embedding, implement a matrix multiply that you will replace, add layernorm, derive attention from scratch, stack heads, add the residual, write the loss, compute the gradient by hand, then let autodiff check your arithmetic.
The rule throughout is that nothing appears that could not be written by the reader in NumPy. Module six swaps from toy data to a real corpus and shows the loss curve bending. By the last lesson the reader has a model that generates text and, more importantly, an opinion about why it works.
How it's built
Content is Markdown with YAML front matter, built by MkDocs Material into a static site. Math goes through KaTeX, diagrams through Mermaid, and code through the Material highlighting pipeline with a filename bar and a copy button. Everything is plain files in a repository, which means the course diffs like code - and it does, because the lessons are versioned against the reference implementation they describe.
The publishing path is a GitHub Actions workflow that builds on push and deploys the site to GitHub Pages. There is no server, no database, and nothing to maintain.
Status
Public and complete at six modules and thirty-three lessons. The reference implementation and the lessons live in the same repository so they cannot drift apart.
Credits
- Andrej Karpathy, for MicroGPT and for the standard of explanation it set
- Vaswani et al. (2017), Attention Is All You Need
- Rupesh Srivastava's work on the minimal training setups this course imitates
- MkDocs Material
