Lecture notes
Math 4377/6308 – Advanced Linear Algebra I
Vaughn Climenhaga
December 3, 2013
,2
The primary text for this course is “Linear Algebra and its Applications”,
second edition, by Peter D. Lax (hereinafter referred to as [Lax]). The
lectures will follow the presentation in this book, and many of the homework
exercises will be taken from it.
You may occasionally find it helpful to have access to other resources
that give a more expanded and detailed presentation of various topics than
is available in Lax’s book or in the lecture notes. To this end I suggest the
following list of external references, which are freely available online.
(Bee) “A First Course in Linear Algebra”, by Robert A. Beezer, University
of Puget Sound. Long and comprehensive (1027 pages). Starts from
the very beginning: vectors and matrices as arrays of numbers, systems
of equations, row reduction. Organisation of book is a little non-
standard: chapters and sections are given abbreviations instead of
numbers. http://linear.ups.edu/
(CDW) “Linear Algebra”, by David Cherney, Tom Denton, and Andrew
Waldron, UC Davis. 308 pages. Covers similar material to [Bee].
https://www.math.ucdavis.edu/~linear/
(Hef ) “Linear Algebra”, by Jim Hefferon, Saint Michael’s College. 465
pages. Again, starts from the very beginning. http://joshua.smcvt.
edu/linearalgebra/
(LNS) “Linear Algebra as an Introduction to Abstract Mathematics”, by
Isaiah Lankham, Bruno Nachtergaele, and Anne Schilling, UC Davis.
247 pages. More focused on abstraction than the previous three ref-
erences, and hence somewhat more in line with the present course.
https://www.math.ucdavis.edu/~anne/linear_algebra/
(Tre) “Linear Algebra Done Wrong”,1 by Sergei Treil, Brown University.
276 pages. Starts from the beginning but also takes a more abstract
view. http://www.math.brown.edu/~treil/papers/LADW/LADW.html
The books listed above can all be obtained freely via the links provided.
(These links are also on the website for this course.) Another potentially
useful resource is the series of video lectures by Gilbert Strang from MIT’s
Open CourseWare project: http://ocw.mit.edu/courses/mathematics/
18-06-linear-algebra-spring-2010/video-lectures/
1
If the title seems strange, it may help to be aware that there is a relatively famous
textbook by Sheldon Axler called “Linear Algebra Done Right”, which takes a different
approach to linear algebra than do many other books, including the ones here.
,Lecture 1 Monday, Aug. 26
Motivation, linear spaces, and isomorphisms
Further reading: [Lax] Ch. 1 (p. 1–4). See also [Bee] p. 317–333; [CDW]
Ch. 5 (p. 79–87); [Hef ] Ch. 2 (p. 76–87); [LNS] Ch. 4 (p. 36–40); [Tre]
Ch. 1 (p. 1–5)
1.1 General motivation
We begin by mentioning a few examples that on the surface may not appear
to have anything to do with linear algebra, but which turn out to involve
applications of the machinery we will develop in this course. These (and
other similar examples) serve as a motivation for many of the things that
we do.
1. Fibonacci sequence. The Fibonacci sequence is the sequence of
numbers 1, 1, 2, 3, 5, 8, 13, . . . , where each number is the sum of the
previous two. We can use linear algebra to find an exact formula for
the nth term. Somewhat surprisingly, it has the odd-looking form
√ !n √ !n !
1 1+ 5 1− 5
√ − .
5 2 2
We will discuss this example when we talk about eigenvalues, eigen-
vectors, and diagonalisation.
2. Google. Linear algebra and Markov chain methods are at the heart
of the PageRank algorithm that was central to Google’s early success
as an internet search engine. We will discuss this near the end of the
course.
3. Multivariable calculus. In single-variable calculus, the derivative is
a number, while in multivariable calculus it is a matrix. The proper
way to understand this is that in both cases, the derivative is a linear
transformation. We will reinforce this point of view throughout the
course.
4. Singular value decomposition. This is an important tool that has
applications to image compression, suggestion algorithms such as those
used by Netflix, and many other areas. We will mention these near
the end of the course, time permitting.
3
, 4 LECTURE 1. MONDAY, AUG. 26
5. Rotations. Suppose I start with a sphere, and rotate it first around
one axis (through whatever angle I like) and then around a different
axis (again through whatever angle I like). How does the final position
of the sphere relate to the initial one? Could I have gotten from start to
finish by doing a single rotation around a single axis? How would that
axis relate to the axes I actually performed rotations around? This
and other questions in three-dimensional geometry can be answered
using linear algebra, as we will see later.
6. Partial differential equations. Many important problems in ap-
plied mathematics and engineering can be formulated as partial dif-
ferential equations; the heat equation and the wave equation are two
fundamental examples. A complete theory of PDEs requires functional
analysis, which considers vector spaces whose elements are not arrays
of numbers (as in Rn ), but rather functions with certain differentiabil-
ity properties.
There are many other examples: to chemistry (vibrations of molecules in
terms of their symmetries), integration techniques in calculus (partial frac-
tions), magic squares, error-correcting codes, etc.
1.2 Background: general mathematical notation
and terminology
Throughout this course we will assume a working familiarity with standard
mathematical notation and terminology. Some of the key pieces of back-
ground are reviewed on the first assignment, which is due at the beginning
of the next lecture.
For example, recall that the symbol R stands for the set of real numbers;
C stands for the set of complex numbers; Z stands for the integers (both
positive and negative); and N stands for the natural numbers 1, 2, 3, . . . . Of
particular importance will be the use of the quantifiers ∃ (“there exists”) and
∀ (“for all”), which will appear in many definitions and theorems throughout
the course.
Example 1.1. 1. The statement “∃x ∈ R such that x + 2 = 7” is true,
because we can choose x = 5.
2. The statement “x + 2 = 7 ∀x ∈ R” is false, because x + 2 6= 7 when
x 6= 5.
Math 4377/6308 – Advanced Linear Algebra I
Vaughn Climenhaga
December 3, 2013
,2
The primary text for this course is “Linear Algebra and its Applications”,
second edition, by Peter D. Lax (hereinafter referred to as [Lax]). The
lectures will follow the presentation in this book, and many of the homework
exercises will be taken from it.
You may occasionally find it helpful to have access to other resources
that give a more expanded and detailed presentation of various topics than
is available in Lax’s book or in the lecture notes. To this end I suggest the
following list of external references, which are freely available online.
(Bee) “A First Course in Linear Algebra”, by Robert A. Beezer, University
of Puget Sound. Long and comprehensive (1027 pages). Starts from
the very beginning: vectors and matrices as arrays of numbers, systems
of equations, row reduction. Organisation of book is a little non-
standard: chapters and sections are given abbreviations instead of
numbers. http://linear.ups.edu/
(CDW) “Linear Algebra”, by David Cherney, Tom Denton, and Andrew
Waldron, UC Davis. 308 pages. Covers similar material to [Bee].
https://www.math.ucdavis.edu/~linear/
(Hef ) “Linear Algebra”, by Jim Hefferon, Saint Michael’s College. 465
pages. Again, starts from the very beginning. http://joshua.smcvt.
edu/linearalgebra/
(LNS) “Linear Algebra as an Introduction to Abstract Mathematics”, by
Isaiah Lankham, Bruno Nachtergaele, and Anne Schilling, UC Davis.
247 pages. More focused on abstraction than the previous three ref-
erences, and hence somewhat more in line with the present course.
https://www.math.ucdavis.edu/~anne/linear_algebra/
(Tre) “Linear Algebra Done Wrong”,1 by Sergei Treil, Brown University.
276 pages. Starts from the beginning but also takes a more abstract
view. http://www.math.brown.edu/~treil/papers/LADW/LADW.html
The books listed above can all be obtained freely via the links provided.
(These links are also on the website for this course.) Another potentially
useful resource is the series of video lectures by Gilbert Strang from MIT’s
Open CourseWare project: http://ocw.mit.edu/courses/mathematics/
18-06-linear-algebra-spring-2010/video-lectures/
1
If the title seems strange, it may help to be aware that there is a relatively famous
textbook by Sheldon Axler called “Linear Algebra Done Right”, which takes a different
approach to linear algebra than do many other books, including the ones here.
,Lecture 1 Monday, Aug. 26
Motivation, linear spaces, and isomorphisms
Further reading: [Lax] Ch. 1 (p. 1–4). See also [Bee] p. 317–333; [CDW]
Ch. 5 (p. 79–87); [Hef ] Ch. 2 (p. 76–87); [LNS] Ch. 4 (p. 36–40); [Tre]
Ch. 1 (p. 1–5)
1.1 General motivation
We begin by mentioning a few examples that on the surface may not appear
to have anything to do with linear algebra, but which turn out to involve
applications of the machinery we will develop in this course. These (and
other similar examples) serve as a motivation for many of the things that
we do.
1. Fibonacci sequence. The Fibonacci sequence is the sequence of
numbers 1, 1, 2, 3, 5, 8, 13, . . . , where each number is the sum of the
previous two. We can use linear algebra to find an exact formula for
the nth term. Somewhat surprisingly, it has the odd-looking form
√ !n √ !n !
1 1+ 5 1− 5
√ − .
5 2 2
We will discuss this example when we talk about eigenvalues, eigen-
vectors, and diagonalisation.
2. Google. Linear algebra and Markov chain methods are at the heart
of the PageRank algorithm that was central to Google’s early success
as an internet search engine. We will discuss this near the end of the
course.
3. Multivariable calculus. In single-variable calculus, the derivative is
a number, while in multivariable calculus it is a matrix. The proper
way to understand this is that in both cases, the derivative is a linear
transformation. We will reinforce this point of view throughout the
course.
4. Singular value decomposition. This is an important tool that has
applications to image compression, suggestion algorithms such as those
used by Netflix, and many other areas. We will mention these near
the end of the course, time permitting.
3
, 4 LECTURE 1. MONDAY, AUG. 26
5. Rotations. Suppose I start with a sphere, and rotate it first around
one axis (through whatever angle I like) and then around a different
axis (again through whatever angle I like). How does the final position
of the sphere relate to the initial one? Could I have gotten from start to
finish by doing a single rotation around a single axis? How would that
axis relate to the axes I actually performed rotations around? This
and other questions in three-dimensional geometry can be answered
using linear algebra, as we will see later.
6. Partial differential equations. Many important problems in ap-
plied mathematics and engineering can be formulated as partial dif-
ferential equations; the heat equation and the wave equation are two
fundamental examples. A complete theory of PDEs requires functional
analysis, which considers vector spaces whose elements are not arrays
of numbers (as in Rn ), but rather functions with certain differentiabil-
ity properties.
There are many other examples: to chemistry (vibrations of molecules in
terms of their symmetries), integration techniques in calculus (partial frac-
tions), magic squares, error-correcting codes, etc.
1.2 Background: general mathematical notation
and terminology
Throughout this course we will assume a working familiarity with standard
mathematical notation and terminology. Some of the key pieces of back-
ground are reviewed on the first assignment, which is due at the beginning
of the next lecture.
For example, recall that the symbol R stands for the set of real numbers;
C stands for the set of complex numbers; Z stands for the integers (both
positive and negative); and N stands for the natural numbers 1, 2, 3, . . . . Of
particular importance will be the use of the quantifiers ∃ (“there exists”) and
∀ (“for all”), which will appear in many definitions and theorems throughout
the course.
Example 1.1. 1. The statement “∃x ∈ R such that x + 2 = 7” is true,
because we can choose x = 5.
2. The statement “x + 2 = 7 ∀x ∈ R” is false, because x + 2 6= 7 when
x 6= 5.
