Matrix algebra for beginners, Part I
matrices, determinants, inverses
Jeremy Gunawardena
Department of Systems Biology
Harvard Medical School
200 Longwood Avenue, Cambridge, MA 02115, USA
3 January 2006
Contents
1 Introduction 1
2 Systems of linear equations 1
3 Matrices and matrix multiplication 2
4 Matrices and complex numbers 5
5 Can we use matrices to solve linear equations? 6
6 Determinants and the inverse matrix 7
7 Solving systems of linear equations 9
8 Properties of determinants 10
9 Gaussian elimination 11
1
,1 Introduction
This is a Part I of an introduction to the matrix algebra needed for the Harvard Systems Biology
101 graduate course. Molecular systems are inherently many dimensional—there are usually many
molecular players in any biological system—and linear algebra is a fundamental tool for thinking
about many dimensional systems. It is also widely used in other areas of biology and science.
I will describe the main concepts needed for the course—determinants, matrix inverses, eigenvalues
and eigenvectors—and try to explain where the concepts come from, why they are important and
how they are used. If you know some of the material already, you may find the treatment here quite
slow. There are mostly no proofs but there are worked examples in low dimensions. New concepts
appear in italics when they are introduced or defined and there is an index of important items at
the end. There are many textbooks on matrix algebra and you should refer to one of these for more
details, if you need them.
Thanks to Matt Thomson for spotting various bugs. Any remaining errors are my responsibility.
Let me know if you come across any or have any comments.
2 Systems of linear equations
Matrices first arose from trying to solve systems of linear equations. Such problems go back to the
very earliest recorded instances of mathematical activity. A Babylonian tablet from around 300 BC
states the following problem1 :
There are two fields whose total area is 1800 square yards. One produces grain at the
rate of 2/3 of a bushel per square yard while the other produces grain at the rate of 1/2
a bushel per square yard. If the total yield is 1100 bushels, what is the size of each field?
If we let x and y stand for the areas of the two fields in square yards, then the problem amounts to
saying that
x+y = 1800
(1)
2x/3 + y/2 = 1100 .
This is a system of two linear equations in two unknowns. The linear refers to the fact that the
unknown quantities appear just as x and y, not as 1/x or y 3 . Equations with the latter terms are
nonlinear and their study forms part of a different branch of mathematics, called algebraic geometry.
Generally speaking, it is much harder to say anything about nonlinear equations. However, linear
equations are a different matter: we know a great deal about them. You will, of course, have seen
examples like (1) before and will know how to solve them. (So what is the answer?). Let us consider
a more general problem (this is the kind of thing mathematicians love to do) in which we do not
know exactly what the coefficients are (ie: 1, 2/3, 1/2, 1800, 1100):
ax + by = u
(2)
cx + dy = v,
and suppose, just to keep things simple, that none of the numbers a, b, c or d are 0. You should be
able to solve this too so let us just recall how to do it. If we multiply the first equation by (c/a),
which we can do because a 6= 0, and subtract the second, we find that
(cb/a)y − dy = cu/a − v .
1 For an informative account of the history of matrices and determinants, see
http://www-groups.dcs.st-and.ac.uk/ history/HistTopics/Matrices and determinants.html.
1
, Provided (ad − bc) 6= 0, we can divide across and find that
av − cu
y= . (3)
ad − bc
Similarly, we find that
ud − bv
x= . (4)
ad − bc
The quantity (ad − bc), which we did not notice in the Babylonian example above, turns out to
be quite important. It is a determinant. If it is non-zero, then the system of equations (2) always
has a unique solution: the determinant determines whether a solution exists, hence the name. You
might check that it is indeed non-zero for example (1). If the determinant is zero, the situation gets
more interesting, which is the mathematician’s way of saying that it gets a lot more complicated.
Depending on u, v, the system may have no solution at all or it may have many solutions. You
should be able to find some examples to convince yourself of these assertions.
One of the benefits of looking at a more general problem, like (2) instead of (1), is that you often learn
something, like the importance of determinants, that was hard to see in the more concrete problem.
Let us take this a step further (generalisation being an obsessive trait among mathematicians). How
would you solve a system of 3 equations with 3 unknowns,
ax + by + cz = u
dx + ey + f z = v (5)
gx + hy + iz = w,
or, more generally still, a system of n equations with n unknowns?
You think this is going too far? In 1800, when Carl Friedrich Gauss was trying to calculate the
orbit of the asteroid Pallas, he came up against a system of 6 linear equations in 6 unknowns.
(Astronomy, like biology, also has lots of moving parts.) Gauss was a great mathematician—perhaps
the greatest—and one of his very minor accomplishments was to work out a systematic version of the
technique we used above for solving linear systems of arbitrary size. It is called Gaussian elimination
in his honour. However, it was later discovered that the “Nine Chapters of the Mathematical Art”,
a handbook of practical mathematics (surveying, rates of exchange, fair distribution of goods, etc)
written in China around the 3rd century BC, uses the same method on simple examples. Gauss
made the method into what we would now call an algorithm: a systematic procedure that can be
applied to any system of equations. We will learn more about Gaussian elimination in §9 below.
The modern way to solve a system of linear equations is to transform the problem from one about
numbers and ordinary algebra into one about matrices and matrix algebra. This turns out to be
a very powerful idea but we will first need to know some basic facts about matrices before we can
understand how they help to solve linear equations.
3 Matrices and matrix multiplication
A matrix is any rectangular array of numbers. If the array has n rows and m columns, then it is an
n×m matrix. The numbers n and m are called the dimensions of the matrix. We will usually denote
matrices with capital letters, like A, B, etc, although we will sometimes use lower case letters for
one dimensional matrices (ie: 1 × m or n × 1 matrices). One dimensional matrices are often called
vectors, as in row vector for a n × 1 matrix or column vector for a 1 × m matrix but we are going
to use the word “vector” to refer to something different in Part II. We will use the notation Aij to
refer to the number in the i-th row and j-th column. For instance, we can extract the numerical
coefficients from the system of linear equations in (5) and represent them in the matrix
a b c
A= d e f . (6)
g h i
2
matrices, determinants, inverses
Jeremy Gunawardena
Department of Systems Biology
Harvard Medical School
200 Longwood Avenue, Cambridge, MA 02115, USA
3 January 2006
Contents
1 Introduction 1
2 Systems of linear equations 1
3 Matrices and matrix multiplication 2
4 Matrices and complex numbers 5
5 Can we use matrices to solve linear equations? 6
6 Determinants and the inverse matrix 7
7 Solving systems of linear equations 9
8 Properties of determinants 10
9 Gaussian elimination 11
1
,1 Introduction
This is a Part I of an introduction to the matrix algebra needed for the Harvard Systems Biology
101 graduate course. Molecular systems are inherently many dimensional—there are usually many
molecular players in any biological system—and linear algebra is a fundamental tool for thinking
about many dimensional systems. It is also widely used in other areas of biology and science.
I will describe the main concepts needed for the course—determinants, matrix inverses, eigenvalues
and eigenvectors—and try to explain where the concepts come from, why they are important and
how they are used. If you know some of the material already, you may find the treatment here quite
slow. There are mostly no proofs but there are worked examples in low dimensions. New concepts
appear in italics when they are introduced or defined and there is an index of important items at
the end. There are many textbooks on matrix algebra and you should refer to one of these for more
details, if you need them.
Thanks to Matt Thomson for spotting various bugs. Any remaining errors are my responsibility.
Let me know if you come across any or have any comments.
2 Systems of linear equations
Matrices first arose from trying to solve systems of linear equations. Such problems go back to the
very earliest recorded instances of mathematical activity. A Babylonian tablet from around 300 BC
states the following problem1 :
There are two fields whose total area is 1800 square yards. One produces grain at the
rate of 2/3 of a bushel per square yard while the other produces grain at the rate of 1/2
a bushel per square yard. If the total yield is 1100 bushels, what is the size of each field?
If we let x and y stand for the areas of the two fields in square yards, then the problem amounts to
saying that
x+y = 1800
(1)
2x/3 + y/2 = 1100 .
This is a system of two linear equations in two unknowns. The linear refers to the fact that the
unknown quantities appear just as x and y, not as 1/x or y 3 . Equations with the latter terms are
nonlinear and their study forms part of a different branch of mathematics, called algebraic geometry.
Generally speaking, it is much harder to say anything about nonlinear equations. However, linear
equations are a different matter: we know a great deal about them. You will, of course, have seen
examples like (1) before and will know how to solve them. (So what is the answer?). Let us consider
a more general problem (this is the kind of thing mathematicians love to do) in which we do not
know exactly what the coefficients are (ie: 1, 2/3, 1/2, 1800, 1100):
ax + by = u
(2)
cx + dy = v,
and suppose, just to keep things simple, that none of the numbers a, b, c or d are 0. You should be
able to solve this too so let us just recall how to do it. If we multiply the first equation by (c/a),
which we can do because a 6= 0, and subtract the second, we find that
(cb/a)y − dy = cu/a − v .
1 For an informative account of the history of matrices and determinants, see
http://www-groups.dcs.st-and.ac.uk/ history/HistTopics/Matrices and determinants.html.
1
, Provided (ad − bc) 6= 0, we can divide across and find that
av − cu
y= . (3)
ad − bc
Similarly, we find that
ud − bv
x= . (4)
ad − bc
The quantity (ad − bc), which we did not notice in the Babylonian example above, turns out to
be quite important. It is a determinant. If it is non-zero, then the system of equations (2) always
has a unique solution: the determinant determines whether a solution exists, hence the name. You
might check that it is indeed non-zero for example (1). If the determinant is zero, the situation gets
more interesting, which is the mathematician’s way of saying that it gets a lot more complicated.
Depending on u, v, the system may have no solution at all or it may have many solutions. You
should be able to find some examples to convince yourself of these assertions.
One of the benefits of looking at a more general problem, like (2) instead of (1), is that you often learn
something, like the importance of determinants, that was hard to see in the more concrete problem.
Let us take this a step further (generalisation being an obsessive trait among mathematicians). How
would you solve a system of 3 equations with 3 unknowns,
ax + by + cz = u
dx + ey + f z = v (5)
gx + hy + iz = w,
or, more generally still, a system of n equations with n unknowns?
You think this is going too far? In 1800, when Carl Friedrich Gauss was trying to calculate the
orbit of the asteroid Pallas, he came up against a system of 6 linear equations in 6 unknowns.
(Astronomy, like biology, also has lots of moving parts.) Gauss was a great mathematician—perhaps
the greatest—and one of his very minor accomplishments was to work out a systematic version of the
technique we used above for solving linear systems of arbitrary size. It is called Gaussian elimination
in his honour. However, it was later discovered that the “Nine Chapters of the Mathematical Art”,
a handbook of practical mathematics (surveying, rates of exchange, fair distribution of goods, etc)
written in China around the 3rd century BC, uses the same method on simple examples. Gauss
made the method into what we would now call an algorithm: a systematic procedure that can be
applied to any system of equations. We will learn more about Gaussian elimination in §9 below.
The modern way to solve a system of linear equations is to transform the problem from one about
numbers and ordinary algebra into one about matrices and matrix algebra. This turns out to be
a very powerful idea but we will first need to know some basic facts about matrices before we can
understand how they help to solve linear equations.
3 Matrices and matrix multiplication
A matrix is any rectangular array of numbers. If the array has n rows and m columns, then it is an
n×m matrix. The numbers n and m are called the dimensions of the matrix. We will usually denote
matrices with capital letters, like A, B, etc, although we will sometimes use lower case letters for
one dimensional matrices (ie: 1 × m or n × 1 matrices). One dimensional matrices are often called
vectors, as in row vector for a n × 1 matrix or column vector for a 1 × m matrix but we are going
to use the word “vector” to refer to something different in Part II. We will use the notation Aij to
refer to the number in the i-th row and j-th column. For instance, we can extract the numerical
coefficients from the system of linear equations in (5) and represent them in the matrix
a b c
A= d e f . (6)
g h i
2
