MA106 Linear Algebra lecture notes
Lecturers: Martin Bright and Daan Krammer
Warwick, January 2011
Contents
1 Number systems and fields 3
1.1 Axioms for number systems . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Vector spaces 4
2.1 Examples of vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 Linear independence, spanning and bases of vector spaces 6
3.1 Linear dependence and independence . . . . . . . . . . . . . . . . . . . . 6
3.2 Spanning vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Bases of vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 Existence of a basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4 Subspaces 10
5 Linear transformations 13
5.1 Definition and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.2 Kernels and images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.3 Rank and nullity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.4 Operations on linear maps . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6 Matrices 18
7 Linear transformations and matrices 20
7.1 Setting up the correspondence . . . . . . . . . . . . . . . . . . . . . . . . 20
7.2 The correspondence between operations on linear maps and matrices . . 22
7.3 Linear equations and the inverse image problem . . . . . . . . . . . . . . 24
8 Elementary operations and the rank of a matrix 25
8.1 Gauss transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8.2 Elementary row operations . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.3 The augmented matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.4 Row reducing a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.5 Elementary column operations . . . . . . . . . . . . . . . . . . . . . . . 29
8.6 The rank of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.7 The rank criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9 The inverse of a linear transformation and of a matrix 33
9.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.2 Matrix inversion by row reduction . . . . . . . . . . . . . . . . . . . . . 34
9.3 Elementary matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
9.4 Application to linear equations . . . . . . . . . . . . . . . . . . . . . . . 36
1
,10 The determinant of a matrix 37
10.1 Definition of the determinant . . . . . . . . . . . . . . . . . . . . . . . . 37
10.2 The effect of matrix operations on the determinant . . . . . . . . . . . . 39
10.3 The determinant of a product . . . . . . . . . . . . . . . . . . . . . . . . 42
10.4 Minors and cofactors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
10.5 The inverse of a matrix using determinants . . . . . . . . . . . . . . . . 44
10.6 Cramer’s rule for solving simultaneous equations . . . . . . . . . . . . . 45
11 Change of basis and equivalent matrices 45
12 Similar matrices, eigenvectors and eigenvalues 48
12.1 Similar matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
12.2 Eigenvectors and eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . 49
12.3 The scalar product – symmetric and orthogonal matrices . . . . . . . . . 54
2
,1 Number systems and fields
We introduce the number systems most commonly used in mathematics.
1. The natural numbers N = {1, 2, 3, 4, . . .}.
In N, addition is possible but not subtraction; e.g. 2 − 3 6∈ N.
2. The integers Z = {. . . , −2, −1, 0, 1, 2, 3, . . .}.
In Z, addition, subtraction and multiplication are always possible, but not division;
e.g. 2/3 6∈ Z.
3. The rational numbers Q = {p/q | p, q ∈ Z, q 6= 0}.
In Q, addition, subtraction,
√ multiplication and division (except by zero) are all
possible. However, 2 6∈ Q.
4. The real numbers R. These are the numbers which can be expressed as decimals.
The rational numbers are those with finite or recurring decimals.
In R, addition, subtraction, multiplication and division (except
√ by zero) are still
possible, and all positive numbers have square roots, but −1 6∈ R.
5. The complex numbers C = {x + iy | x, y ∈ R}, where i2 = −1.
In C, addition, subtraction, multiplication and division (except by zero) are still
possible, and all, numbers have square roots. In fact all polynomial equations with
coefficients in C have solutions in C.
1.1 Axioms for number systems
Laws governing the way numbers combine together are called axioms. Any particular
axiom might be true in some number systems but not in others.
Axioms for addition. Let S be a number system.
A1. α + β = β + α for all α, β ∈ S.
A2. (α + β) + γ = α + (β + γ) for all α, β, γ ∈ S.
A3. There is a number 0 ∈ S such that α + 0 = 0 + α = α for all α ∈ S.
A4. For each number α ∈ S there exists a number −α ∈ S such that α + (−α) =
(−α) + α = 0.
These axioms may or may not be satisfied by a given number system S. For example,
in N, A1 and A2 hold but A3 and A4 do not hold. A1–A4 all hold in Z, Q, R and C.
Axioms for multiplication.
M1. α.β = β.α for all α, β ∈ S.
M2. (α.β).γ = α.(β.γ) for all α, β, γ ∈ S.
M3. There is a number 1 ∈ S such that α.1 = 1.α = α for all α ∈ S.
M4. For each number α ∈ S with α 6= 0, there exists a number α−1 ∈ S such that
α.α−1 = α−1 .α = 1.
In N and Z, M1–M3 hold but M4 does not hold. M1–M4 all hold in Q, R and C.
3
, Axiom relating addition and multiplication.
D. (α + β).γ = α.γ + β.γ for all α, β, γ ∈ S.
Definition. A set S on which addition and multiplication are defined is called a field if
it satisfies each of the axioms A1, A2, A3, A4, M1, M2, M3, M4, D, and if, in addition,
1 6= 0.
Roughly speaking, S is a field if addition, subtraction, multiplication and division
(except by zero) are all possible in S. We shall always use the letter K for a general
field.
Example. N and Z are not fields, but Q, R and C are all fields.
There are many other fields, including some finite fields. For example, for each prime
number p, there is a field Fp = {0, 1, 2, . . . , p − 1} with p elements, where addition and
multiplication are carried out modulo p. Thus, in F7 , we have 5 + 4 = 2, 5 × 4 = 6 and
5−1 = 3 because 5 × 3 = 1. The smallest such field F2 has just two elements 0 and 1,
where 1+1 = 0. This field is extremely important in Computer Science since an element
of F2 represents a bit of information.
Various other familiar properties of numbers, such as 0α = 0, (−α)β = −(αβ) =
α(−β), (−α)(−β) = αβ, (−1)α = −α, for all α, β ∈ S, can be proved from the axioms.
Why would we want to do this, when we can see they’re true anyway? The point is
that, when we meet a new number system, it is enough to check whether the axioms
hold; if they do, then all these properties follow automatically.
However, occasionally you need to be careful. For example, in F2 we have 1 + 1 = 0,
and so it is not possible to divide by 2 in this field.
2 Vector spaces
Definition. A vector space over a field K is a set V which has two basic operations,
addition and scalar multiplication, satisfying certain requirements. Thus for every pair
u, v ∈ V , u + v ∈ V is defined, and for every α ∈ K, αv ∈ V is defined. For V to
be called a vector space, the following axioms must be satisfied for all α, β ∈ K and all
u, v ∈ V .
(i) Vector addition satisfies axioms A1, A2, A3 and A4.
(ii) α(u + v) = αu + αv;
(iii) (α + β)v = αv + βv;
(iv) (αβ)v = α(βv);
(v) 1v = v.
Elements of the field K will be called scalars. Note that we will use boldface letters
like v to denote vectors. The zero vector in V will be written as 0V , or usually just 0.
This is different from the zero scalar 0 = 0K ∈ K.
For nearly all results in this course, there is no loss in assuming that K is the field R
of real numbers. So you may assume this if you find it helpful to do so. Just occasionally,
we will need to assume K = C the field of complex numbers.
However, it is important to note that nearly all arguments in Linear Algebra use
only the axioms for a field and so are valid for any field, which is why shall use a general
field K for most of the course.
4
Lecturers: Martin Bright and Daan Krammer
Warwick, January 2011
Contents
1 Number systems and fields 3
1.1 Axioms for number systems . . . . . . . . . . . . . . . . . . . . . . . . . 3
2 Vector spaces 4
2.1 Examples of vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . 5
3 Linear independence, spanning and bases of vector spaces 6
3.1 Linear dependence and independence . . . . . . . . . . . . . . . . . . . . 6
3.2 Spanning vectors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.3 Bases of vector spaces . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7
3.4 Existence of a basis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4 Subspaces 10
5 Linear transformations 13
5.1 Definition and examples . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
5.2 Kernels and images . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
5.3 Rank and nullity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
5.4 Operations on linear maps . . . . . . . . . . . . . . . . . . . . . . . . . . 17
6 Matrices 18
7 Linear transformations and matrices 20
7.1 Setting up the correspondence . . . . . . . . . . . . . . . . . . . . . . . . 20
7.2 The correspondence between operations on linear maps and matrices . . 22
7.3 Linear equations and the inverse image problem . . . . . . . . . . . . . . 24
8 Elementary operations and the rank of a matrix 25
8.1 Gauss transformations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25
8.2 Elementary row operations . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.3 The augmented matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26
8.4 Row reducing a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
8.5 Elementary column operations . . . . . . . . . . . . . . . . . . . . . . . 29
8.6 The rank of a matrix . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30
8.7 The rank criterion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9 The inverse of a linear transformation and of a matrix 33
9.1 Definitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33
9.2 Matrix inversion by row reduction . . . . . . . . . . . . . . . . . . . . . 34
9.3 Elementary matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35
9.4 Application to linear equations . . . . . . . . . . . . . . . . . . . . . . . 36
1
,10 The determinant of a matrix 37
10.1 Definition of the determinant . . . . . . . . . . . . . . . . . . . . . . . . 37
10.2 The effect of matrix operations on the determinant . . . . . . . . . . . . 39
10.3 The determinant of a product . . . . . . . . . . . . . . . . . . . . . . . . 42
10.4 Minors and cofactors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42
10.5 The inverse of a matrix using determinants . . . . . . . . . . . . . . . . 44
10.6 Cramer’s rule for solving simultaneous equations . . . . . . . . . . . . . 45
11 Change of basis and equivalent matrices 45
12 Similar matrices, eigenvectors and eigenvalues 48
12.1 Similar matrices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48
12.2 Eigenvectors and eigenvalues . . . . . . . . . . . . . . . . . . . . . . . . 49
12.3 The scalar product – symmetric and orthogonal matrices . . . . . . . . . 54
2
,1 Number systems and fields
We introduce the number systems most commonly used in mathematics.
1. The natural numbers N = {1, 2, 3, 4, . . .}.
In N, addition is possible but not subtraction; e.g. 2 − 3 6∈ N.
2. The integers Z = {. . . , −2, −1, 0, 1, 2, 3, . . .}.
In Z, addition, subtraction and multiplication are always possible, but not division;
e.g. 2/3 6∈ Z.
3. The rational numbers Q = {p/q | p, q ∈ Z, q 6= 0}.
In Q, addition, subtraction,
√ multiplication and division (except by zero) are all
possible. However, 2 6∈ Q.
4. The real numbers R. These are the numbers which can be expressed as decimals.
The rational numbers are those with finite or recurring decimals.
In R, addition, subtraction, multiplication and division (except
√ by zero) are still
possible, and all positive numbers have square roots, but −1 6∈ R.
5. The complex numbers C = {x + iy | x, y ∈ R}, where i2 = −1.
In C, addition, subtraction, multiplication and division (except by zero) are still
possible, and all, numbers have square roots. In fact all polynomial equations with
coefficients in C have solutions in C.
1.1 Axioms for number systems
Laws governing the way numbers combine together are called axioms. Any particular
axiom might be true in some number systems but not in others.
Axioms for addition. Let S be a number system.
A1. α + β = β + α for all α, β ∈ S.
A2. (α + β) + γ = α + (β + γ) for all α, β, γ ∈ S.
A3. There is a number 0 ∈ S such that α + 0 = 0 + α = α for all α ∈ S.
A4. For each number α ∈ S there exists a number −α ∈ S such that α + (−α) =
(−α) + α = 0.
These axioms may or may not be satisfied by a given number system S. For example,
in N, A1 and A2 hold but A3 and A4 do not hold. A1–A4 all hold in Z, Q, R and C.
Axioms for multiplication.
M1. α.β = β.α for all α, β ∈ S.
M2. (α.β).γ = α.(β.γ) for all α, β, γ ∈ S.
M3. There is a number 1 ∈ S such that α.1 = 1.α = α for all α ∈ S.
M4. For each number α ∈ S with α 6= 0, there exists a number α−1 ∈ S such that
α.α−1 = α−1 .α = 1.
In N and Z, M1–M3 hold but M4 does not hold. M1–M4 all hold in Q, R and C.
3
, Axiom relating addition and multiplication.
D. (α + β).γ = α.γ + β.γ for all α, β, γ ∈ S.
Definition. A set S on which addition and multiplication are defined is called a field if
it satisfies each of the axioms A1, A2, A3, A4, M1, M2, M3, M4, D, and if, in addition,
1 6= 0.
Roughly speaking, S is a field if addition, subtraction, multiplication and division
(except by zero) are all possible in S. We shall always use the letter K for a general
field.
Example. N and Z are not fields, but Q, R and C are all fields.
There are many other fields, including some finite fields. For example, for each prime
number p, there is a field Fp = {0, 1, 2, . . . , p − 1} with p elements, where addition and
multiplication are carried out modulo p. Thus, in F7 , we have 5 + 4 = 2, 5 × 4 = 6 and
5−1 = 3 because 5 × 3 = 1. The smallest such field F2 has just two elements 0 and 1,
where 1+1 = 0. This field is extremely important in Computer Science since an element
of F2 represents a bit of information.
Various other familiar properties of numbers, such as 0α = 0, (−α)β = −(αβ) =
α(−β), (−α)(−β) = αβ, (−1)α = −α, for all α, β ∈ S, can be proved from the axioms.
Why would we want to do this, when we can see they’re true anyway? The point is
that, when we meet a new number system, it is enough to check whether the axioms
hold; if they do, then all these properties follow automatically.
However, occasionally you need to be careful. For example, in F2 we have 1 + 1 = 0,
and so it is not possible to divide by 2 in this field.
2 Vector spaces
Definition. A vector space over a field K is a set V which has two basic operations,
addition and scalar multiplication, satisfying certain requirements. Thus for every pair
u, v ∈ V , u + v ∈ V is defined, and for every α ∈ K, αv ∈ V is defined. For V to
be called a vector space, the following axioms must be satisfied for all α, β ∈ K and all
u, v ∈ V .
(i) Vector addition satisfies axioms A1, A2, A3 and A4.
(ii) α(u + v) = αu + αv;
(iii) (α + β)v = αv + βv;
(iv) (αβ)v = α(βv);
(v) 1v = v.
Elements of the field K will be called scalars. Note that we will use boldface letters
like v to denote vectors. The zero vector in V will be written as 0V , or usually just 0.
This is different from the zero scalar 0 = 0K ∈ K.
For nearly all results in this course, there is no loss in assuming that K is the field R
of real numbers. So you may assume this if you find it helpful to do so. Just occasionally,
we will need to assume K = C the field of complex numbers.
However, it is important to note that nearly all arguments in Linear Algebra use
only the axioms for a field and so are valid for any field, which is why shall use a general
field K for most of the course.
4
