Three rules about columns let you compute a determinant of any size. You expand a 3x3 by hand once, see why software uses elimination instead, and learn the product rule that makes determinants useful for chains of transformations.
About 55 minutes
By the end you can
Use the three column rules: swapping, scaling, and adding a multiple of one column to another.
Compute a 3x3 determinant by cofactor expansion, and explain why software uses elimination instead.
Use det(AB)=detAdetB to reason about chains of transformations.
The last lesson gave the determinant a meaning, the factor by which a transformation scales area, and a formula for 2x2 matrices, ad−bc. The weight matrices inside a neural network have hundreds or thousands of rows, and a 2x2 formula does not stretch to them by itself. Yet np.linalg.det returns the determinant of a 1000×1000 matrix in well under a second. How?
The answer rests on three rules about what happens to the determinant when you change the columns. This lesson builds up from those rules: first the rules and their pictures, then a recipe for 3x3 matrices that you will do by hand once, then the method software actually uses, and last the rule for chains of transformations.
Every method for computing determinants, by hand or by machine, rests on three rules about changing the columns. Each has a picture, so start there.
The widget shows the example from the last lesson, A=[3112], with determinant 5. As before, the blue and coral arrows are the columns A^=(3,1) and A^=(1,2), with tips you can drag; the white parallelogram is the image of the unit square; the det A readout gives its signed area, coral when negative. You can also type into the matrix cells.
Here are the three rules in words, each checked with the formula on the same matrix.
Swapping the two columns flips the sign. The turn from one side of the parallelogram to the other reverses. [1231] has (1)(1)−(3)(2)=1−6=−5.
Scaling one column scales the determinant by the same number. One side of the parallelogram stretches, so its area stretches with it. Doubling the first column gives [6212], with (6)(2)−(1)(2)=12−2=10=2×5.
Adding a multiple of one column to another changes nothing. This slides one side of the parallelogram along the direction of the other: a shear, which keeps both the base and the height. Adding the first column to the second gives the new second column (1,2)+(3,1)=(4,3), and [3143] has (3)(3)−(4)(1)=9−4=5.
The next exercise proves that the three rules hold for every 2x2 matrix, not only this one.
On paperThe three column rules
Let A=[acbd], with columns (a,c) and (b,d). Using only detA=ad−bc, show that:
Swapping the two columns flips the sign of the determinant.
Multiplying the first column by a number k multiplies the determinant by k.
Adding k times the first column to the second column leaves the determinant unchanged.
Then check each rule separately, each time starting from [2113], whose determinant is 5: (a) swap its columns; (b) triple its first column; (c) add 2 times its first column to its second. Compute the three new determinants and enter them in the order (a), (b), (c).
Work it on real paper: writing each step is the point. Then check your final answer here and compare your working with the walk-through.
The three new determinants, (a), (b), (c)
One entry per box, top to bottom. 0.25, -2, 3/4 and sqrt(2) all work. Enter moves to the next empty box and checks once all are filled.
The column rules hold in any number of dimensions, and so does the picture. This section shows how both grow, starting with 3D, and gives a recipe you will use by hand once.
In 3D the unit cube lands on a slanted box (a parallelepiped) spanned by the three columns, and the determinant is its signed volume. The sign is negative when the box is a mirror image of the cube, the way a left hand mirrors a right hand. Zero means the three columns lie in one plane, or on one line, so all of space is flattened.
Minors. Write the entries of a 3x3 matrix as aij for row i and column j. Delete the row and the column of one entry, aij, and a smaller 2x2 matrix is left; call it Mij. Its determinant, detMij, is called the minor of aij. (Some books use the word minor for the smaller matrix itself. Either way, the recipe below needs its determinant.) For example, in
A=211031122,
deleting row 1 and column 1 (the row and column of the top-left entry, 2) leaves M11=[3122], so the minor of that entry is (3)(2)−(2)(1)=4.
The cofactor expansion along the first row builds a 3x3 determinant from three 2x2 ones: each entry of the first row times its minor, with signs +,−,+:
An entry's minor with its sign attached is called its cofactor, which gives the method its name. For the example, one term at a time:
First term, with sign +: 2det[3122]=2((3)(2)−(2)(1))=2(6−2)=2×4=8.
Second term, with sign −: the entry is 0, so −0⋅det[1122]=0, whatever the minor is.
Third term, with sign +: 1det[1131]=1((1)(1)−(3)(1))=1−3=−2.
Add them: detA=8−0−2=6.
The zero entry saved a whole 2x2 determinant. You may expand along any row or column instead, as long as the sign of the entry in row i and column j is (−1)i+j: plus on the diagonal, then alternating like a checkerboard,
+−+−+−+−+.
So pick the row or column with the most zeros. The same recipe works for any size: an n×n determinant is an alternating sum of n entries, each times a determinant of size n−1,
detA=j=1∑n(−1)1+ja1jdetM1j,
where M1j is A with row 1 and column j deleted, and ∑j=1n means "add up the following for j=1,2,…,n".
Where does it come from? From the column rules, extended to n dimensions. The determinant is the only function of the columns that is linear in each column separately (scaling one column scales it, and splitting one column into a sum of two splits it into a sum of two determinants), flips sign when two columns swap, and gives 1 for the identity matrix; the expansion is what those rules force. The box below shows how for a 3x3.
Go slower: Where the expansion comes from, for a 3x3
Two facts do the work. First, the three rules hold for rows as well as columns, because a matrix and its transpose have the same determinant (the next section checks this for a 2x2). Second, the determinant is linear in each row: scaling a row scales the determinant, and if two matrices are identical except in row 1, the matrix whose row 1 is the sum of their first rows has the sum of their determinants. For a 2x2 you can check it directly: (a+a′)d−(b+b′)c=(ad−bc)+(a′d−b′c).
Step 1, split the first row. Write (a11,a12,a13)=(a11,0,0)+(0,a12,0)+(0,0,a13). By linearity, detA is the sum of three determinants. Each belongs to a matrix with one nonzero entry in its first row and the same rows 2 and 3 as A.
Step 2, clear the column under that entry. Take the middle piece,
P=0a21a31a12a22a320a23a33.
If a12=0, then P has a row of zeros and its determinant is 0, which matches the term a12detM12=0. Otherwise subtract a22/a12 times row 1 from row 2, and a32/a12 times row 1 from row 3. Row 1 is zero outside column 2, so these row shears change only the column-2 entries of rows 2 and 3, and they turn those entries into zeros. Shears do not change the determinant:
detP=det0a21a31a12000a23a33.
Step 3, move that column to the front. Swapping columns 1 and 2 flips the sign:
detP=−deta12000a21a310a23a33.
Step 4, read off the volume. This last matrix sends ^ to a12^, and it sends the other two basis vectors into the plane of the second and third axes, where it acts exactly as the minor M12=[a21a31a23a33]. So the unit cube becomes a box whose base is the parallelogram of M12 in that plane, with signed area detM12, and whose remaining edge, a12^, stands perpendicular to the base. Volume is base times height, so the determinant is a12detM12 (a negative a12 or a negative detM12 each flip the box, and the signs multiply). The middle term of the expansion is therefore −a12detM12.
The signs. The first piece needs no swap, so its term is +a11detM11. For the third piece, move column 3 to the front one neighbor at a time: swap it with column 2, then with column 1. That is two swaps, so the sign is +, and the other two columns keep their original order, which is the order the minor M13 uses. In an n×n matrix, moving column j to the front this way takes j−1 swaps, which is where the sign (−1)1+j comes from.
To recap: the expansion turns one n×n determinant into n determinants of size n−1, with alternating signs, and it follows from the column rules. Practice it once by hand, then once in code.
Work it outA 3x3 determinant
Compute det1302−14021 by cofactor expansion. Pick a row or column that saves work, and enter a whole number.
det
Type a number: 0.25, -2, 3/4 and sqrt(2) all work. Enter checks.
Now turn the same recipe into a program. The function calls itself on the minors until they are small enough to finish directly.
Code itDeterminants by cofactor expansion
Turn the cofactor expansion into code, the way you did it on paper.
det2(M) returns ad−bc for a 2x2 matrix.
minor(M, i, j) returns a new list of lists: M with row i and column j removed.
det(M) computes the determinant of any square matrix by expanding along the first row and calling itself on the minors. It raises ValueError if M is not square.
Matrices arrive as lists of lists, and Python counts rows and columns from 0, so the lesson's M1j is minor(M, 0, j - 1) here. The tests compare with np.linalg.det and check the product rule from later in this lesson.
⌘+Enter runsPython sleeps until you run code
Write your code where the starter says raise NotImplementedError, then press Run tests. Each check says what it expects.
Your cofactor code is correct for any size, so why does nobody use it on big matrices? Count the work.
Expanding an n×n determinant needs n determinants of size n−1, each of those needs n−1 of size n−2, and so on down to size 1: on the order of n! multiplications, where n!=n×(n−1)×⋯×2×1. (The exact count, from the code below, is about 1.7×n!.) For n=20, 20! alone is about 2.4×1018, and the full count is over 4×1018 multiplications. A computer doing a billion multiplications per second would need over a century.
Software uses elimination instead, which is the column rules applied to rows. (Rows obey the same rules because a matrix and its transpose, the matrix with rows and columns swapped, have the same determinant. For a 2x2, det[abcd]=ad−cb, the same number as ad−bc.)
Subtract multiples of one row from the rows below it until every entry under the diagonal is zero. Each step is a shear, so the determinant does not change.
If the next diagonal position holds a zero, swap in a row from below. Each swap flips the sign. If every entry from there down is zero, the columns are dependent and the determinant is 0. (Software swaps even when the entry is merely small, choosing the largest one available, for accuracy. The next lesson says why.)
The matrix is now upper triangular (zeros everywhere below the diagonal), and the determinant of a triangular matrix is the product of its diagonal. For a 2x2, det[a0bd]=ad−b⋅0=ad. For bigger ones, expand down the first column, where only the top entry can be nonzero, and repeat.
On the example from the last section, three shears do it.
Row 2 minus 21 of row 1: (1,3,2)−21(2,0,1)=(1−1,3−0,2−0.5)=(0,3,1.5).
Row 3 minus 21 of row 1: (1,1,2)−21(2,0,1)=(1−1,1−0,2−0.5)=(0,1,1.5).
Row 3 minus 31 of the new row 2: (0,1,1.5)−31(0,3,1.5)=(0−0,1−1,1.5−0.5)=(0,0,1).
No swaps were needed, so no sign changes, and the answer matches the cofactor expansion. Elimination costs about n3/3 multiplications: under 3,000 for n=20. np.linalg.det does exactly this in compiled code (an LU factorization, which is elimination with the multipliers saved). Run the comparison:
⌘+Enter runs · edit freelyPython sleeps until you run code
Read the table. For a 3x3 the two methods cost about the same (9 multiplications against 13). By n=10 the cofactor expansion needs over six million multiplications and elimination needs 384. The chart plots the base-10 logarithm of each count, so each step up by 1 on its vertical axis is ten times more work: the cofactor curve keeps bending upward, while the elimination curve flattens out.
Networks chain one transformation after another, so the last question is what happens to area along a chain.
Do B first, then A. From module 2, the combined transformation is the single matrix AB. Every area is scaled by detB in the first step and then by detA in the second, so
det(AB)=detAdetB.
The signs work out too: two flips make no flip, and (−1)(−1)=1. A check with numbers: A=[1324] has detA=(1)(4)−(2)(3)=4−6=−2, and B=[3111] has detB=(3)(1)−(1)(1)=3−1=2. Each entry of the product is a row of A times a column of B:
AB=[1⋅3+2⋅13⋅3+4⋅11⋅1+2⋅13⋅1+4⋅1]=[51337],det(AB)=(5)(7)−(3)(13)=35−39=−4=(−2)(2).Go slower: The same fact by algebra, for 2x2 matrices
Let A=[acbd] and B=[egfh]. Then
AB=[ae+bgce+dgaf+bhcf+dh].
Its determinant is (ae+bg)(cf+dh)−(af+bh)(ce+dg). Expand each product:
(ae+bg)(cf+dh)=acef+adeh+bcfg+bdgh,(af+bh)(ce+dg)=acef+adfg+bceh+bdgh.
Subtract. The terms acef and bdgh appear in both, so they cancel:
det(AB)=adeh+bcfg−adfg−bceh.
Group the terms that contain ad and the terms that contain bc:
det(AB)=ad(eh−fg)−bc(eh−fg).
Both groups share the factor eh−fg; take it out:
det(AB)=(ad−bc)(eh−fg)=detAdetB.
A handful of useful facts follow, each in one line.
det(BA)=det(AB), even though BA and AB are usually different matrices: both determinants are detAdetB.
det(Ak)=(detA)k for k applications in a row.
If A can be undone by a matrix A−1 (the next lesson), then det(A−1)=1/detA. The reason: A−1A=I, so det(A−1)detA=detI=1; divide both sides by detA.
det(cA)=cndetA for an n×n matrix, not cdetA. Multiplying the matrix by c multiplies each of its n columns by c, and each column contributes its own factor. Doubling a 3x3 matrix multiplies volumes by 23=8.
There is no such rule for sums. det(A+B) is usually not detA+detB: with A=B=I in 2D, det(2I)=4, not 1+1=2.
A product is singular exactly when at least one of its factors is. In a chain of transformations, one collapse anywhere flattens the whole chain, and no later step can recover what was lost.
Test which of these rules apply, and which are the classic slips.
Quick checkWhat the product rule says, and what it does not
A and B are 2x2 matrices with detA=3 and detB=−2. Which statements must be true? Select all that apply.
Software computes determinants for you, but the idea keeps coming back in this course. In the next lesson the determinant becomes the test for whether a matrix can be undone. In module 4 it finds eigenvalues: the equation det(A−λI)=0 asks for which numbers λ the matrix A−λI squashes some direction to zero (directions that do not turn). In module 5 the determinant of a Jacobian measures how a curved map scales tiny areas near a point (Jacobians and the chain rule).
Before moving on, a practice set mixes every kind of determinant question from this lesson: 2x2, 3x3, products and the scaling rules. Work until the streak feels routine.
Practice setDeterminant reps
3 correct in a row completes the set. A miss starts the count again.
A is a 2×2 matrix with detA=3. Find det(A⊤).
det
Type a number: 0.25, -2, 3/4 and sqrt(2) all work. Enter checks.