Module 5 · 6 lessons · about 360 minutes
Derivatives, gradients and Jacobians
Derivatives as local linear maps, gradients as the direction of steepest climb, Jacobians and the chain rule as matrix multiplication, and gradient descent.
- Lessons
- 0/6
- Exercises
- 0/25
- Code
- 0/6
- Checkpoint
- Not taken
Already comfortable with this? Take the checkpoint cold. If you pass, move on and come back only for what you missed.

- 1Derivatives as zoomZoom in far enough on a smooth curve and it becomes a straight line. The slope of that line is the derivative, and the approximation it gives is the idea every training algorithm runs on.65 min
- 2Derivatives by experimentThe definition of the derivative doubles as a way to measure one. Forward and central differences, why the central one is far more accurate, and why making the step too small backfires in floating point. This is gradient checking.35 min
- 3GradientsWith many inputs, the derivative becomes a vector with one entry per input. It points in the direction the function climbs fastest, and every training step moves against it.65 min
- 4Jacobians and the chain ruleWhen a function takes a vector and returns a vector, its derivative is a matrix, the Jacobian. Composing functions multiplies their Jacobians, and backprop is that product computed from the loss end, one vector at a time.80 min
- 5Gradient descentStep against the gradient and repeat. Why that works, exactly how large a step can be before training blows up, why narrow valleys make it crawl, and how momentum speeds it up.70 min
- 6Mini-batches and linear regressionA real training loss is an average over many examples, so training estimates its gradient from a small random batch, and Adam gives each weight its own step size. Then fit linear regression both ways, exactly and by descent, and see why the scale of the features decides how fast descent gets there.45 min
- Checkpoint11 questions. Pass with 80% to finish the module.
Practice sets for this module
Fresh problems every time. Use them until the mechanics feel automatic.
Let . Find .
Type a number: 0.25, -2, 3/4 and sqrt(2) all work. Enter checks.
Let . Find the directional derivative at in the direction of the unit vector . Give the exact value.
Type a number: 0.25, -2, 3/4 and sqrt(2) all work. Enter checks.
Let . Find the Jacobian at .
One entry per box, row by row (Tab moves along a row). 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box and checks once all are filled.