ml.lab
Python sleeps until you run code
03 Determinants, inverses and rank

Lesson 3 of 4

Inverses and systems

The inverse runs a transformation backwards when that is possible. In practice you solve $\mathbf{A}\mathbf{x} = \mathbf{b}$ by elimination instead of forming the inverse, and nearly singular matrices show why the answer can still be fragile.

About 60 minutes
By the end you can
  • Derive and use the 2x2 inverse formula, and say exactly when an inverse exists.

  • Solve Ax=b\mathbf{A}\mathbf{x} = \mathbf{b} by elimination, and explain why numerical code solves rather than inverts.

  • Explain in plain words why nearly singular matrices amplify small errors, and what the condition number measures.

Two people talk at once and two microphones record them. Microphone 1 sits nearer the first speaker and picks up 2 parts of voice 1 for every 1 part of voice 2. Microphone 2 hears both voices equally. If the voices are x1x_1 and x2x_2 at some instant, the microphones read

b1=2x1+x2,b2=x1+x2,that is,b=Ax with A=[2111].b_1 = 2x_1 + x_2, \qquad b_2 = x_1 + x_2, \qquad\text{that is,}\qquad \mathbf{b} = \mathbf{A}\mathbf{x} \text{ with } \mathbf{A} = \begin{bmatrix} 2 & 1 \\ 1 & 1 \end{bmatrix}.

At one instant the microphones read b=(5,3)\mathbf{b} = (5,\allowbreak 3). What were the voices? You need to run the mixing backwards.

Now move microphone 2 so that it hears exactly half as much of each voice as microphone 1: its row becomes (1,0.5)(1,\allowbreak 0.5). Then b2b_2 is always exactly half of b1b_1. The second microphone adds nothing new, and no amount of cleverness recovers two voices from what is really one recording. That matrix, [2110.5]\begin{bmatrix} 2 & 1 \\ 1 & 0.5 \end{bmatrix}, has determinant (2)(0.5)−(1)(1)=0(2)(0.5) - (1)(1) = 0: it is singular, the fatal case from the determinant lesson.

This lesson is about the first situation: how to undo a transformation when that is possible, how computers actually do it, and why some matrices that can be undone are still dangerous to undo. The second situation, measuring what a singular matrix loses, is the next lesson, on rank and on LoRA.

Undoing a transformation

Before any formula, what does "undo" mean for a matrix?

The inverse of a square matrix A\mathbf{A}, written A−1\mathbf{A}^{-1}, is the matrix that undoes it: A−1(Ax)=x\mathbf{A}^{-1}(\mathbf{A}\mathbf{x}) = \mathbf{x} for every x\mathbf{x}. Doing A\mathbf{A} and then A−1\mathbf{A}^{-1} is doing nothing, so A−1A=I\mathbf{A}^{-1}\mathbf{A} = \mathbf{I}, the identity matrix (ones on the diagonal, zeros elsewhere: the matrix that moves nothing). For square matrices the other order works too: AA−1=I\mathbf{A}\mathbf{A}^{-1} = \mathbf{I}. In the transformation widget, the inverse is the morph played backwards.

Some inverses you can guess from the picture:

  • Scaling by 2 is undone by scaling by 12\tfrac12.
  • Rotating by an angle θ\theta is undone by rotating by −θ-\theta.
  • The shear [1101]\begin{bmatrix} 1 & 1 \\ 0 & 1 \end{bmatrix} slides each point to the right by its height; [1−101]\begin{bmatrix} 1 & -1 \\ 0 & 1 \end{bmatrix} slides it back.
  • A reflection is undone by doing it again.

The microphone matrix is not one you can guess, so try it in the widget. It answers the question "does some matrix send the recordings back to the voices?" The blue and coral arrows are the columns of the matrix in the cells, and you can drag their tips. The white dashed arrow is an input v\mathbf{v}, which you can also drag, and the solid white arrow is its image Av\mathbf{A}\mathbf{v} (the readout gives its coordinates). The dashed blue-then-coral path shows Av\mathbf{A}\mathbf{v} built as v1v_1 copies of column 1 plus v2v_2 copies of column 2. The widget starts with the mixing matrix and the voices v=(2,1)\mathbf{v} = (2,\allowbreak 1).

When does an inverse exist? Exactly when det⁡A≠0\det\mathbf{A} \ne 0. Both directions follow from the determinant lesson.

  • If det⁡A=0\det\mathbf{A} = 0, then A\mathbf{A} flattens space, so whole lines of inputs share each output. Given such an output, there is no way to choose which input to return, so no inverse exists.
  • If det⁡A≠0\det\mathbf{A} \ne 0, nothing is flattened. For a 2x2 matrix, the formula in the next section builds the inverse explicitly. For larger matrices, elimination (later in this lesson) finds the input that lands on each basis vector, and never gets stuck when the determinant is not zero; those inputs are the columns of A−1\mathbf{A}^{-1}, as step 4 of the walkthrough showed.

The inverse also has to scale areas back to where they were, so det⁡(A−1)=1/det⁡A\det(\mathbf{A}^{-1}) = 1/\det\mathbf{A}. The mixing matrix and its inverse both have determinant 1, and 1/1=11/1 = 1.

The 2x2 inverse

Guessing works for rotations and shears. For any 2x2 matrix there is a formula you can use by hand. For A=[abcd]\mathbf{A} = \begin{bmatrix} a & b \\ c & d \end{bmatrix} with ad−bc≠0ad - bc \ne 0,

A−1=1ad−bc[d−b−ca].\mathbf{A}^{-1} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}.

In words: swap the diagonal entries, negate the off-diagonal ones, divide every entry by the determinant. The division is exactly where the formula breaks when the determinant is zero.

Go slower: Deriving the 2x2 inverse

The first column of A−1\mathbf{A}^{-1} is A−1ı^\mathbf{A}^{-1}\hat{\imath}: the input that A\mathbf{A} sends to ı^=(1,0)\hat{\imath} = (1,\allowbreak 0). The second column is the input that A\mathbf{A} sends to ȷ^=(0,1)\hat{\jmath} = (0,\allowbreak 1). So we solve two small systems.

First column. Find (x1,x2)(x_1,\allowbreak x_2) with ax1+bx2=1(1)cx1+dx2=0(2)a x_1 + b x_2 = 1 \quad (1) \qquad\qquad c x_1 + d x_2 = 0 \quad (2)

To get rid of x2x_2, make its coefficient the same in both equations. Multiply (1) by dd:  adx1+bdx2=d\ adx_1 + bdx_2 = d. Multiply (2) by bb:  bcx1+bdx2=0\ bcx_1 + bdx_2 = 0. Subtract the second from the first; the bdx2bdx_2 terms cancel: (ad−bc) x1=d.(ad - bc)\,x_1 = d. To get rid of x1x_1 instead, multiply (1) by cc:  acx1+bcx2=c\ acx_1 + bcx_2 = c. Multiply (2) by aa:  acx1+adx2=0\ acx_1 + adx_2 = 0. Subtract the first from the second; the acx1acx_1 terms cancel: (ad−bc) x2=−c.(ad - bc)\,x_2 = -c.

Second column. Find (y1,y2)(y_1,\allowbreak y_2) with ay1+by2=0(3)cy1+dy2=1(4)a y_1 + b y_2 = 0 \quad (3) \qquad\qquad c y_1 + d y_2 = 1 \quad (4)

The same two moves. To get rid of y2y_2, multiply (3) by dd:  ady1+bdy2=0\ ady_1 + bdy_2 = 0. Multiply (4) by bb:  bcy1+bdy2=b\ bcy_1 + bdy_2 = b. Subtract the second from the first; the bdy2bdy_2 terms cancel: (ad−bc) y1=−b.(ad - bc)\,y_1 = -b. To get rid of y1y_1, multiply (3) by cc:  acy1+bcy2=0\ acy_1 + bcy_2 = 0. Multiply (4) by aa:  acy1+ady2=a\ acy_1 + ady_2 = a. Subtract the first from the second; the acy1acy_1 terms cancel: (ad−bc) y2=a.(ad - bc)\,y_2 = a.

Assemble. Divide each result by ad−bcad - bc (allowed only when it is not zero), and put the two columns side by side: A−1=[x1y1x2y2]=1ad−bc[d−b−ca].\mathbf{A}^{-1} = \begin{bmatrix} x_1 & y_1 \\ x_2 & y_2 \end{bmatrix} = \frac{1}{ad - bc}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix}.

Check. Multiply A\mathbf{A} by the matrix in the brackets, one row times one column at a time: [abcd][d−b−ca]=[ad−bc−ab+bacd−dc−cb+da]=[ad−bc00ad−bc]=(ad−bc) I,\begin{bmatrix} a & b \\ c & d \end{bmatrix}\begin{bmatrix} d & -b \\ -c & a \end{bmatrix} = \begin{bmatrix} ad - bc & -ab + ba \\ cd - dc & -cb + da \end{bmatrix} = \begin{bmatrix} ad - bc & 0 \\ 0 & ad - bc \end{bmatrix} = (ad - bc)\,\mathbf{I}, so dividing by ad−bcad - bc gives exactly I\mathbf{I}.

Back to the microphones, one move at a time. Here a=2a = 2, b=1b = 1, c=1c = 1, d=1d = 1.

  1. The determinant: ad−bc=(2)(1)−(1)(1)=2−1=1ad - bc = (2)(1) - (1)(1) = 2 - 1 = 1. It is not zero, so the inverse exists.
  2. Swap aa and dd, and negate bb and cc: [1−1−12]\begin{bmatrix} 1 & -1 \\ -1 & 2 \end{bmatrix}.
  3. Divide every entry by the determinant, 1, which changes nothing: A−1=[1−1−12]\mathbf{A}^{-1} = \begin{bmatrix} 1 & -1 \\ -1 & 2 \end{bmatrix}.
  4. Apply it to the recordings, one row at a time:
A−1[53]=[(1)(5)+(−1)(3)(−1)(5)+(2)(3)]=[5−3−5+6]=[21].\mathbf{A}^{-1}\begin{bmatrix} 5 \\ 3 \end{bmatrix} = \begin{bmatrix} (1)(5) + (-1)(3) \\ (-1)(5) + (2)(3) \end{bmatrix} = \begin{bmatrix} 5 - 3 \\ -5 + 6 \end{bmatrix} = \begin{bmatrix} 2 \\ 1 \end{bmatrix}.

The voices were x1=2x_1 = 2 and x2=1x_2 = 1, as the widget showed. Check by mixing them again: 2(2)+1=52(2) + 1 = 5 and 2+1=32 + 1 = 3. The second arrangement, [2110.5]\begin{bmatrix} 2 & 1 \\ 1 & 0.5 \end{bmatrix}, has determinant 0, so step 3 would divide by zero, and there is no inverse.

Now invert one yourself. This matrix has a negative determinant, which is where most slips happen.

Work it outInvert a 2x2

Find A−1\mathbf{A}^{-1} for A=[4231]\mathbf{A} = \begin{bmatrix} 4 & 2 \\ 3 & 1 \end{bmatrix} with the swap, negate, divide recipe. Enter the four entries row by row; decimals or fractions (such as -1/2) both work. Before you check, multiply A\mathbf{A} by your answer and see whether you get I\mathbf{I}.

A^{-1}

One entry per box, row by row (Tab moves along a row). 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box and checks once all are filled.

A practice set builds the habit: swap, negate, divide, then check by multiplying. About half the items ask which input lands on a given output instead; apply the inverse to that output, as you did for the microphones.

Practice setInverse reps
3 correct in a row completes the set. A miss starts the count again.

Solve Ax=y\mathbf{A}\mathbf{x} = \mathbf{y} for A=[−2−523]\mathbf{A} = \begin{bmatrix}-2 & -5 \\ 2 & 3\end{bmatrix} and y=[−106]\mathbf{y} = \begin{bmatrix}-10 \\ 6\end{bmatrix}. In words: which input x\mathbf{x} does A\mathbf{A} send to y\mathbf{y}?

x

One entry per box, top to bottom. 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box and checks once all are filled.

In short: an inverse exists exactly when the determinant is not zero, and for a 2x2 matrix it is "swap, negate, divide by the determinant".

Solving Ax=b\mathbf{A}\mathbf{x} = \mathbf{b}

The microphone question is really a system of equations, and for large matrices nobody writes down the inverse to answer it. This section shows the method people and computers actually use.

The question is: which input lands on b\mathbf{b}? There are two ways to picture it.

  • Column picture. From module 2, Ax=x1(column 1)+x2(column 2)\mathbf{A}\mathbf{x} = x_1(\text{column 1}) + x_2(\text{column 2}). So solving means finding the combination of the columns that makes b\mathbf{b}, exactly the coefficient problem from module 1. For the microphones, 2(2,1)+1(1,1)=(4,2)+(1,1)=(5,3)2(2,\allowbreak 1) + 1(1,\allowbreak 1) = (4,\allowbreak 2) + (1,\allowbreak 1) = (5,\allowbreak 3). This is the dashed path from the widget.
  • Row picture. Each equation is a line in the (x1,x2)(x_1,\allowbreak x_2) plane: 2x1+x2=52x_1 + x_2 = 5 and x1+x2=3x_1 + x_2 = 3. The solution is the point on both lines, where they cross, at (2,1)(2,\allowbreak 1). When det⁡A=0\det\mathbf{A} = 0 the lines are parallel: either they never meet (no solution) or they are the same line (infinitely many).

The inverse gives a formula, x=A−1b\mathbf{x} = \mathbf{A}^{-1}\mathbf{b}, but the method people and computers use is elimination: subtract a multiple of one equation from another to remove a variable, repeat until the system is triangular (each equation has one fewer unknown than the one above it), then solve from the bottom up, which is called back substitution. For the microphones:

  1. The equations are (1) 2x1+x2=52x_1 + x_2 = 5 and (2) x1+x2=3x_1 + x_2 = 3.
  2. To remove x1x_1 from (2), subtract the right multiple of (1). Equation (2) has 1 x11\,x_1 and equation (1) has 2 x12\,x_1, so the multiplier is 12\tfrac12. Half of (1) is x1+12x2=52x_1 + \tfrac12 x_2 = \tfrac52.
  3. Subtract it from (2), term by term:
(x1−x1)+(x2−12x2)=3−52⟹12x2=12⟹x2=1.(x_1 - x_1) + (x_2 - \tfrac12 x_2) = 3 - \tfrac52 \quad\Longrightarrow\quad \tfrac12 x_2 = \tfrac12 \quad\Longrightarrow\quad x_2 = 1.
  1. Back substitution: put x2=1x_2 = 1 into (1). Then 2x1+1=52x_1 + 1 = 5, so 2x1=42x_1 = 4 and x1=2x_1 = 2.

Why is this allowed? Combining true equations gives true equations, so every solution of the old system solves the new one. And each step can be undone by adding the multiple back, so the new system has no extra solutions either. The solution never changes, while the matrix becomes triangular. These are the same row shears that computed determinants in the last lesson.

One practical detail. If a zero sits where you need to divide, swap two equations. In fact software swaps whenever a larger entry is available lower down in the same column (partial pivoting), because dividing by a tiny number blows up rounding errors. You will see that happen in the code exercise below.

Now solve a new system both ways, and check it with the column picture.

On paperSolve a system two ways

Solve the system 2x1+x2=4,x1+3x2=72x_1 + x_2 = 4,\allowbreak \qquad x_1 + 3x_2 = 7

  1. By elimination: subtract a multiple of the first equation from the second to remove x1x_1, solve for x2x_2, then substitute back.
  2. With the inverse: write the system as Ax=b\mathbf{A}\mathbf{x} = \mathbf{b}, find A−1\mathbf{A}^{-1} with the 2x2 formula, and compute A−1b\mathbf{A}^{-1}\mathbf{b}.
  3. Check your answer in the column picture: which combination of the columns of A\mathbf{A} makes b\mathbf{b}?

Enter x1x_1 and then x2x_2 in the check.

Work it on real paper: writing each step is the point. Then check your final answer here and compare your working with the walk-through.

x = (x1, x2)

One entry per box, top to bottom. 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box and checks once all are filled.

Solve, do not invert

On paper A−1b\mathbf{A}^{-1}\mathbf{b} and "solve Ax=b\mathbf{A}\mathbf{x} = \mathbf{b}" are the same thing. In numerical code they are not, and np.linalg.solve(A, b) beats np.linalg.inv(A) @ b on both counts that matter.

  • Cost. Solving factors A\mathbf{A} by elimination once, about n3/3n^3/3 multiply-adds for an n×nn \times n matrix, then back-substitutes for about n2n^2 more. Forming the full inverse takes about n3n^3 multiply-adds, roughly three times as much, before you even multiply by b\mathbf{b}. (For many right-hand sides, pass them all as the columns of a matrix: np.linalg.solve(A, B) reuses one factorization.)
  • Accuracy. When A\mathbf{A} is close to singular, the inverse route loses more accuracy. The run below shows it on a famously difficult family of matrices.

The code builds Hilbert matrices of sizes 4 to 12, picks right-hand sides whose exact answer is all ones, and solves each system both ways. For each it prints the condition number (a measure of how close to singular the matrix is, explained in the next section), how well each answer satisfies the equations, and how far each answer is from the truth.

⌘+Enter runs · edit freelyPython sleeps until you run code

Read the columns. The residual ∣Hx−b∣|\mathbf{H}\mathbf{x} - \mathbf{b}| says how well the answer satisfies the equations. With solve it stays below 10−1510^{-15}, the limit of double precision, at every size; with the inverse it grows by many orders of magnitude as the size goes up. The error is the distance from the true answer, and it grows for both methods as the condition number grows. That second effect is not a flaw of either method. It is the matrix itself amplifying rounding errors, and it is the subject of the next section.

Now build both tools yourself: the formula for 2x2 matrices, and elimination for any size.

Code itUndo a matrix: the formula and elimination

Build both ways of undoing a matrix.

  • inverse2(M) returns the inverse of a 2x2 matrix from the formula, as a numpy array, and raises ValueError when the determinant is 0.
  • solve(A, b) solves Ax=b\mathbf{A}\mathbf{x} = \mathbf{b} for any square A\mathbf{A} by Gaussian elimination with partial pivoting, followed by back substitution, without forming an inverse. It must not change its inputs, and it raises ValueError when A\mathbf{A} is singular.

Partial pivoting means: before eliminating below the diagonal in column kk, swap up the row (at or below row kk) whose entry in column kk is largest in absolute value. It fixes zero pivots and, just as important, avoids dividing by tiny ones. One test is designed to catch a solver without it. The tests compare your solve with np.linalg.solve.

⌘+Enter runsPython sleeps until you run code
Write your code where the starter says raise NotImplementedError, then press Run tests. Each check says what it expects.

Nearly singular matrices

In exact arithmetic a matrix is either invertible or not. But real numbers come from measurements and are stored with rounding, and the trouble lives in between: matrices that are invertible, but only just.

An example, step by step. Take A=[1111.001]\mathbf{A} = \begin{bmatrix} 1 & 1 \\ 1 & 1.001 \end{bmatrix} and b=(2,2.001)\mathbf{b} = (2,\allowbreak 2.001). Its determinant is (1)(1.001)−(1)(1)=0.001(1)(1.001) - (1)(1) = 0.001: not zero, but small.

  1. Subtract the first equation from the second: (1−1)x1+(1.001−1)x2=2.001−2(1 - 1)x_1 + (1.001 - 1)x_2 = 2.001 - 2, which leaves 0.001 x2=0.0010.001\,x_2 = 0.001, so x2=1x_2 = 1.
  2. Back substitution: x1+1=2x_1 + 1 = 2, so x1=1x_1 = 1. The solution is x=(1,1)\mathbf{x} = (1,\allowbreak 1).

Now nudge the second measurement by 0.0010.001, to b=(2,2.002)\mathbf{b} = (2,\allowbreak 2.002).

  1. The same subtraction leaves 0.001 x2=2.002−2=0.0020.001\,x_2 = 2.002 - 2 = 0.002, so x2=2x_2 = 2.
  2. Then x1+2=2x_1 + 2 = 2, so x1=0x_1 = 0. The solution is x=(0,2)\mathbf{x} = (0,\allowbreak 2).

A change of 0.001 in one number, about 0.05% of it, moved the answer from (1,1)(1,\allowbreak 1) to (0,2)(0,\allowbreak 2).

The row picture shows why. The two equations are nearly parallel lines, crossing at a very shallow angle, and shifting one of them a hair slides the crossing point a long way along the other.

Two long straight lines crossing at a very shallow angle, with a copy of one of them shifted slightly sideways, and the two crossing points marked far apart along the linesNearly parallel lines: a tiny shift of one line moves the crossing point, the solution, a long way.

The column picture says the same thing another way. The columns (1,1)(1,\allowbreak 1) and (1,1.001)(1,\allowbreak 1.001) point in almost the same direction, so producing a small change in b\mathbf{b} across that direction takes a huge, lopsided combination of them.

The condition number. Say a matrix stretches a direction by the factor ∣Av∣/∣v∣|\mathbf{A}\mathbf{v}| / |\mathbf{v}| for a vector v\mathbf{v} pointing that way. The condition number κ(A)\kappa(\mathbf{A}) (the Greek letter kappa) is the most A\mathbf{A} stretches any direction divided by the least it stretches any direction. For our matrix:

  • The direction (1,1)(1,\allowbreak 1) is stretched by about 2: A(1,1)=(2,2.001)\mathbf{A}(1,\allowbreak 1) = (2,\allowbreak 2.001), about twice as long as (1,1)(1,\allowbreak 1).
  • The direction (1,−1)(1,\allowbreak -1) is almost crushed: A(1,−1)=(1−1, 1−1.001)=(0,−0.001)\mathbf{A}(1,\allowbreak -1) = (1 - 1,\allowbreak \ 1 - 1.001) = (0,\allowbreak -0.001), whose length is 0.001, while (1,−1)(1,\allowbreak -1) has length 2≈1.414\sqrt 2 \approx 1.414. So this direction is stretched by 0.001/1.414≈0.0007070.001 / 1.414 \approx 0.000707: it comes out about 1,400 times shorter.

These two directions alone give a ratio of about 2/0.000707≈2,8002 / 0.000707 \approx 2{,}800. The most and least stretched directions are not exactly these two but lie less than 0.02° away from them, and at the least stretched one the stretch drops to about 0.0005 (with the direction squeezed this hard, even that tiny tilt changes the stretch a lot). The most stretch is still about 2, so κ≈2/0.0005≈4,000\kappa \approx 2 / 0.0005 \approx 4{,}000; computed exactly, κ≈4,002\kappa \approx 4{,}002. Module 4 shows how to find the exact directions.

Why does this ratio matter? A relative error in b\mathbf{b} (the size of the error divided by the size of b\mathbf{b}) can be amplified by up to κ\kappa times in x\mathbf{x}. As a rule of thumb, you lose about log⁡10κ\log_{10}\kappa of the roughly 16 significant digits that double precision carries: here about 3.6 digits. For the Hilbert matrices earlier, κ≈1.6×1013\kappa \approx 1.6 \times 10^{13} at size 10, so about 13 digits are lost, and even solve kept only three or four correct digits.

The run below makes the amplification visible. It adds 300 small random nudges to b\mathbf{b}, solves each time, and plots the nudged right-hand sides next to the solutions they produce.

⌘+Enter runs · edit freelyPython sleeps until you run code

The right-hand sides form a small round cloud around (2,2.001)(2,\allowbreak 2.001). The solutions form a long thin streak through (1,1)(1,\allowbreak 1) along the direction (−1,1)(-1,\allowbreak 1), the direction the matrix barely stretches: a small nudge in b\mathbf{b} needs a big move along that direction to produce it. The worst amplification printed comes out at about the condition number, 4,002, which is the worst case.

As the lesson on computing determinants warned, the determinant does not measure this: 0.1 I0.1\,\mathbf{I} in 100 dimensions has a determinant of 10−10010^{-100} and condition number 1, while a shear with determinant 1 can have a condition number in the millions.

To recap: a nearly singular matrix squeezes some direction almost flat, so undoing it has to blow that direction back up, and any error along it is blown up too. The condition number measures how lopsided the stretching is, and so how much errors can grow.

Now watch the amplification happen by hand, on a slightly different nearly singular system.

Work it outA small nudge, a big move

The matrix A=[2121.01]\mathbf{A} = \begin{bmatrix} 2 & 1 \\ 2 & 1.01 \end{bmatrix} with b=(3,3.01)\mathbf{b} = (3,\allowbreak 3.01) gives x=(1,1)\mathbf{x} = (1,\allowbreak 1). Now the second measurement is off by 0.010.01: b=(3,3.02)\mathbf{b} = (3,\allowbreak 3.02). Solve Ax=b\mathbf{A}\mathbf{x} = \mathbf{b} again, by elimination as in the lesson. Enter x\mathbf{x} as a column: x1x_1 in the top box, x2x_2 in the bottom box. Decimals are fine.

x

One entry per box, top to bottom. 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box and checks once all are filled.

Invertible matrices can be undone, and this lesson showed how, and when to be careful. The next lesson turns to matrices that cannot be undone at all: how many dimensions they keep, what they crush, and why keeping only a few dimensions on purpose is the idea behind LoRA.

Next: Rank and low-rank updates