Linear Algebra in Twenty Five Lectures
Tom Denton and Andrew Waldron
March 27, 2012
Edited by Katrina Glaeser, Rohit Thomas & Travis Scrimshaw
1
,Contents
1 What is Linear Algebra? 12
2 Gaussian Elimination 19
2.1 Notation for Linear Systems . . . . . . . . . . . . . . . . . . . 19
2.2 Reduced Row Echelon Form . . . . . . . . . . . . . . . . . . . 21
3 Elementary Row Operations 27
4 Solution Sets for Systems of Linear Equations 34
4.1 Non-Leading Variables . . . . . . . . . . . . . . . . . . . . . . 35
5 Vectors in Space, n-Vectors 43
5.1 Directions and Magnitudes . . . . . . . . . . . . . . . . . . . . 46
6 Vector Spaces 53
7 Linear Transformations 58
8 Matrices 63
9 Properties of Matrices 72
9.1 Block Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 72
9.2 The Algebra of Square Matrices . . . . . . . . . . . . . . . . 73
10 Inverse Matrix 79
10.1 Three Properties of the Inverse . . . . . . . . . . . . . . . . . 80
10.2 Finding Inverses . . . . . . . . . . . . . . . . . . . . . . . . . . 81
10.3 Linear Systems and Inverses . . . . . . . . . . . . . . . . . . . 82
10.4 Homogeneous Systems . . . . . . . . . . . . . . . . . . . . . . 83
10.5 Bit Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
11 LU Decomposition 88
11.1 Using LU Decomposition to Solve Linear Systems . . . . . . . 89
11.2 Finding an LU Decomposition. . . . . . . . . . . . . . . . . . 90
11.3 Block LDU Decomposition . . . . . . . . . . . . . . . . . . . . 94
2
,12 Elementary Matrices and Determinants 96
12.1 Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
12.2 Elementary Matrices . . . . . . . . . . . . . . . . . . . . . . . 100
13 Elementary Matrices and Determinants II 107
14 Properties of the Determinant 116
14.1 Determinant of the Inverse . . . . . . . . . . . . . . . . . . . . 119
14.2 Adjoint of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . 120
14.3 Application: Volume of a Parallelepiped . . . . . . . . . . . . 122
15 Subspaces and Spanning Sets 124
15.1 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
15.2 Building Subspaces . . . . . . . . . . . . . . . . . . . . . . . . 126
16 Linear Independence 131
17 Basis and Dimension 139
n
17.1 Bases in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
18 Eigenvalues and Eigenvectors 147
18.1 Matrix of a Linear Transformation . . . . . . . . . . . . . . . 147
18.2 Invariant Directions . . . . . . . . . . . . . . . . . . . . . . . . 151
19 Eigenvalues and Eigenvectors II 159
19.1 Eigenspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
20 Diagonalization 165
20.1 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . 165
20.2 Change of Basis . . . . . . . . . . . . . . . . . . . . . . . . . . 166
21 Orthonormal Bases 173
21.1 Relating Orthonormal Bases . . . . . . . . . . . . . . . . . . . 176
22 Gram-Schmidt and Orthogonal Complements 181
22.1 Orthogonal Complements . . . . . . . . . . . . . . . . . . . . 185
23 Diagonalizing Symmetric Matrices 191
3
, 24 Kernel, Range, Nullity, Rank 197
24.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
25 Least Squares 206
A Sample Midterm I Problems and Solutions 211
B Sample Midterm II Problems and Solutions 221
C Sample Final Problems and Solutions 231
D Points Vs. Vectors 256
E Abstract Concepts 258
E.1 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
E.2 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
E.3 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
E.4 Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
E.5 Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
F Sine and Cosine as an Orthonormal Basis 262
G Movie Scripts 264
G.1 Introductory Video . . . . . . . . . . . . . . . . . . . . . . . . 264
G.2 What is Linear Algebra: Overview . . . . . . . . . . . . . . . 265
G.3 What is Linear Algebra: 3 × 3 Matrix Example . . . . . . . . 267
G.4 What is Linear Algebra: Hint . . . . . . . . . . . . . . . . . . 268
G.5 Gaussian Elimination: Augmented Matrix Notation . . . . . . 269
G.6 Gaussian Elimination: Equivalence of Augmented Matrices . . 270
G.7 Gaussian Elimination: Hints for Review Questions 4 and 5 . . 271
G.8 Gaussian Elimination: 3 × 3 Example . . . . . . . . . . . . . . 273
G.9 Elementary Row Operations: Example . . . . . . . . . . . . . 274
G.10 Elementary Row Operations: Worked Examples . . . . . . . . 277
G.11 Elementary Row Operations: Explanation of Proof for Theo-
rem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
G.12 Elementary Row Operations: Hint for Review Question 3 . . 281
G.13 Solution Sets for Systems of Linear Equations: Planes . . . . . 282
G.14 Solution Sets for Systems of Linear Equations: Pictures and
Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
4
Tom Denton and Andrew Waldron
March 27, 2012
Edited by Katrina Glaeser, Rohit Thomas & Travis Scrimshaw
1
,Contents
1 What is Linear Algebra? 12
2 Gaussian Elimination 19
2.1 Notation for Linear Systems . . . . . . . . . . . . . . . . . . . 19
2.2 Reduced Row Echelon Form . . . . . . . . . . . . . . . . . . . 21
3 Elementary Row Operations 27
4 Solution Sets for Systems of Linear Equations 34
4.1 Non-Leading Variables . . . . . . . . . . . . . . . . . . . . . . 35
5 Vectors in Space, n-Vectors 43
5.1 Directions and Magnitudes . . . . . . . . . . . . . . . . . . . . 46
6 Vector Spaces 53
7 Linear Transformations 58
8 Matrices 63
9 Properties of Matrices 72
9.1 Block Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . 72
9.2 The Algebra of Square Matrices . . . . . . . . . . . . . . . . 73
10 Inverse Matrix 79
10.1 Three Properties of the Inverse . . . . . . . . . . . . . . . . . 80
10.2 Finding Inverses . . . . . . . . . . . . . . . . . . . . . . . . . . 81
10.3 Linear Systems and Inverses . . . . . . . . . . . . . . . . . . . 82
10.4 Homogeneous Systems . . . . . . . . . . . . . . . . . . . . . . 83
10.5 Bit Matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84
11 LU Decomposition 88
11.1 Using LU Decomposition to Solve Linear Systems . . . . . . . 89
11.2 Finding an LU Decomposition. . . . . . . . . . . . . . . . . . 90
11.3 Block LDU Decomposition . . . . . . . . . . . . . . . . . . . . 94
2
,12 Elementary Matrices and Determinants 96
12.1 Permutations . . . . . . . . . . . . . . . . . . . . . . . . . . . 97
12.2 Elementary Matrices . . . . . . . . . . . . . . . . . . . . . . . 100
13 Elementary Matrices and Determinants II 107
14 Properties of the Determinant 116
14.1 Determinant of the Inverse . . . . . . . . . . . . . . . . . . . . 119
14.2 Adjoint of a Matrix . . . . . . . . . . . . . . . . . . . . . . . . 120
14.3 Application: Volume of a Parallelepiped . . . . . . . . . . . . 122
15 Subspaces and Spanning Sets 124
15.1 Subspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
15.2 Building Subspaces . . . . . . . . . . . . . . . . . . . . . . . . 126
16 Linear Independence 131
17 Basis and Dimension 139
n
17.1 Bases in R . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142
18 Eigenvalues and Eigenvectors 147
18.1 Matrix of a Linear Transformation . . . . . . . . . . . . . . . 147
18.2 Invariant Directions . . . . . . . . . . . . . . . . . . . . . . . . 151
19 Eigenvalues and Eigenvectors II 159
19.1 Eigenspaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162
20 Diagonalization 165
20.1 Diagonalization . . . . . . . . . . . . . . . . . . . . . . . . . . 165
20.2 Change of Basis . . . . . . . . . . . . . . . . . . . . . . . . . . 166
21 Orthonormal Bases 173
21.1 Relating Orthonormal Bases . . . . . . . . . . . . . . . . . . . 176
22 Gram-Schmidt and Orthogonal Complements 181
22.1 Orthogonal Complements . . . . . . . . . . . . . . . . . . . . 185
23 Diagonalizing Symmetric Matrices 191
3
, 24 Kernel, Range, Nullity, Rank 197
24.1 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 201
25 Least Squares 206
A Sample Midterm I Problems and Solutions 211
B Sample Midterm II Problems and Solutions 221
C Sample Final Problems and Solutions 231
D Points Vs. Vectors 256
E Abstract Concepts 258
E.1 Dual Spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
E.2 Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 258
E.3 Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 259
E.4 Rings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 260
E.5 Algebras . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 261
F Sine and Cosine as an Orthonormal Basis 262
G Movie Scripts 264
G.1 Introductory Video . . . . . . . . . . . . . . . . . . . . . . . . 264
G.2 What is Linear Algebra: Overview . . . . . . . . . . . . . . . 265
G.3 What is Linear Algebra: 3 × 3 Matrix Example . . . . . . . . 267
G.4 What is Linear Algebra: Hint . . . . . . . . . . . . . . . . . . 268
G.5 Gaussian Elimination: Augmented Matrix Notation . . . . . . 269
G.6 Gaussian Elimination: Equivalence of Augmented Matrices . . 270
G.7 Gaussian Elimination: Hints for Review Questions 4 and 5 . . 271
G.8 Gaussian Elimination: 3 × 3 Example . . . . . . . . . . . . . . 273
G.9 Elementary Row Operations: Example . . . . . . . . . . . . . 274
G.10 Elementary Row Operations: Worked Examples . . . . . . . . 277
G.11 Elementary Row Operations: Explanation of Proof for Theo-
rem 3.1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 279
G.12 Elementary Row Operations: Hint for Review Question 3 . . 281
G.13 Solution Sets for Systems of Linear Equations: Planes . . . . . 282
G.14 Solution Sets for Systems of Linear Equations: Pictures and
Explanation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 283
4
