The suite
Where this goes
Part 1 builds the foundations and the first real sequence models. Each later part builds on it, ending with the engineering between a trained model and a product. The first lesson, The big picture, gives a plain-language map of all of it today.
- 01
From matrices to memory
The math refresher and the first real models: vectors, matrices, eigenvalues, Jacobians, backpropagation, recurrent networks and LSTMs trained in your browser.
- Linear algebra
- Calculus for learning
- Backpropagation
- RNNs
- LSTMs
- 02
Attention and transformers
Build a small GPT: tokenization, embeddings and positions, self-attention, the transformer block, training, sampling and the KV cache.
- BPE tokenization
- Self-attention
- Transformer blocks
- A tiny GPT
- Inference
- 03
Pretraining at scale
What changes when the model and data get large: data pipelines, scaling laws, optimizers and schedules, mixed precision and parallelism.
- Data
- Scaling laws
- AdamW and schedules
- Mixed precision
- Parallelism
- 04
Post-training
Turning a base model into an assistant: supervised fine-tuning, reward models, RLHF and DPO, reinforcement learning with verifiable rewards, LoRA and distillation.
- SFT
- Reward models
- RLHF and DPO
- RL with verifiable rewards
- LoRA
- 05
Vision
Convolution, CNNs, detection from sliding windows to the R-CNN family, segmentation with FCN, U-Net and Mask R-CNN, vision transformers and CLIP.
- Convolution
- Detection
- Segmentation
- ViT
- CLIP
- 06
Models in products
Inference, quantization, retrieval, evaluation and agents: the engineering between a trained model and something a customer relies on.
- Serving
- Quantization
- Retrieval
- Evals
- Agents