Part 1

From matrices to memory

Ten modules. The first half rebuilds the math you need, with visual intuition first and hand calculation second. The second half uses it to build networks, recurrent networks and LSTMs from scratch. Each module ends with a checkpoint.

The map

What a language model actually is, what pretraining and post-training mean, where LSTMs and vision models fit, and how to use the exercises here.

The big picture25 min
How this lab works15 min

Vectors and the dot product

Vectors as arrows and as lists, linear combinations, the dot product in algebra and geometry, length, and cosine similarity between embeddings.

Matrices are machines

Matrices as linear transformations, matrix multiplication as composition, and the shapes and batches that every neural network layer is made of.

Determinants, inverses and rank

The determinant as area scaling, inverses as undoing, and rank as how many dimensions survive. The ideas behind LoRA and ill-conditioned training.

The determinant30 min
Inverses, systems and rank35 min

Eigenvectors and eigenvalues

Eigenvectors by picture and by hand, what repeated multiplication does, stability and explosion, symmetric matrices, and the singular value decomposition.

Derivatives, gradients and Jacobians

Derivatives as local linear maps, gradients as the direction of steepest climb, Jacobians and the chain rule as matrix multiplication, and gradient descent.

Neural networks from scratch

Neurons and layers as matrix operations, softmax and cross-entropy, backpropagation derived and implemented, and a full training loop you write yourself.

Recurrent networks

Sequences and hidden state, next-character language models, backpropagation through time, and why plain recurrent networks forget.

LSTMs

The cell state and its gates, an LSTM designed by hand, the full forward and backward pass, training on text in your browser, and your own variants.

From LSTMs to today's models

Attention as a dot-product lookup, how frontier models are pretrained and post-trained, fine-tuning your own model, and how vision models find objects.