Checkpoint
Derivatives, gradients and Jacobians
Eleven questions on the whole module: derivatives and their rules, gradients, Jacobians and the chain rule, and gradient descent. The numbers are new, and several options are the classic mistakes. Work on paper where it helps; the code item is short. You need 80 percent to pass.
- Question 1 of 11.
What is the derivative of at , where is the sigmoid? Give a decimal.
h'(0)Type a number: 0.25, -2, 3/4 and sqrt(2) all work.
- Question 2 of 11.
You check a derivative of a smooth function with a central difference. With the estimate is off by . Ignoring rounding error, about how far off will it be with ? Give a decimal to 3 decimal places.
error with h = 0.05Type a number: 0.25, -2, 3/4 and sqrt(2) all work.
- Question 3 of 11.
A pre-activation passes through either a sigmoid unit or a tanh unit, and the upstream gradient is . How much gradient does each unit pass back to ?
- Question 4 of 11.
The loss on one example is with and . Find at . Enter it as a column, on top.
∇L at w = (1, 1)One entry per box, top to bottom. 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box.
- Question 5 of 11.
At a point, . You take a small step in the unit direction . To first order in the step (the linear approximation), what happens to ?
- Question 6 of 11.
A feed-forward block maps a 768-entry vector up to 3,072 entries with its first layer (Jacobian ), then back down to 768 with its second (Jacobian ). What is the Jacobian of the whole block?
- Question 7 of 11.
Let , with ReLU applied to each entry and . Find the Jacobian of at . Enter the matrix with row for output .
Jacobian at (-1, 1)One entry per box, row by row (Tab moves along a row). 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box.
- Question 8 of 11.
Three logits are equal: . For , what is ? Give a fraction or 3 decimal places.
∂p₁/∂z₂Type a number: 0.25, -2, 3/4 and sqrt(2) all work.
- Question 9 of 11.
Gradient descent on starts at with learning rate . What is ? Give a decimal.
w₂Type a number: 0.25, -2, 3/4 and sqrt(2) all work.
- Question 10 of 11.
Near a minimum, the Hessian has eigenvalues and . You run gradient descent with . What happens?
- Question 11 of 11.
Run the tests before you submit. This question counts as correct if your latest run passes every check.
Code it Write
input_grad(W1, W2, x): the gradient with respect toxof . Use vector-Jacobian products only: never build a Jacobian.Write your code where the starter saysraise NotImplementedError, then press Run tests. Each check says what it expects.