Summary on Linear Algebra
Chapter 1 Linear Equations in Linear Algebra
A linear equation with variables 𝑥1 , … , 𝑥𝑛 is an equation that can be written in the form
𝑎1 𝑥1 + 𝑎2 𝑥2 + ⋯ + 𝑎𝑛 𝑥𝑛 = 𝑏
A system is said to be consistent if it has either one or infinitely many solutions. And is inconsistent if
it has no solution.
Definition Row Equivalent
Two matrices are called row equivalent if there is a sequence of elementary row operations that
transforms one matrix into the other.
Intermezzo
IF the augmented matrices of two linear systems are row equivalent, then the two systems have the
same solution set.
Definition Echelon form and reduced echelon form
A rectangular matrix is in echelon form if it has the following three properties:
1. All nonzero rows are above any zero-rows
2. Each leading entry of a row is in a column to the right of the leading entry of the row above it
3. All entries in a column below a leading entry are zeros
A rectangular matrix is in reduced echelon form if it has the following additional properties:
4. The leading entry in every nonzero row is 1
5. Each leading 1 is the only nonzero entry in its column.
Theorem 1.1 Uniqueness of the Reduced Echelon Form
Each matrix is row equivalent to one and only one reduced echelon matrix.
Definition Pivot position and column
A pivot position in a matrix 𝐴 is a location in 𝐴 that corresponds to a leading 1 in the reduced
echelon form. The pivot column is the column of 𝐴 in which the pivot position is in.
Theorem 1.2 Existence and Uniqueness Theorem
A linear system is consistent if and only if the right most column (the 𝒃-column) of the augmented
matrix is not a pivot column.
If a linear system is consistent, then the solution set either contains an unique solution or infinitely
many.
Intermezzo
A vector equation 𝑥1 𝐚1 + 𝑥2 𝐚2 + ⋯ + 𝑥𝑛 𝐚𝑛 = 𝐛 has the same solution set as the linear system
whose augmented matrix is [𝐚1 𝐚2 ⋯ 𝐚𝑛 𝐛]
,Definition Spanned
If vectors 𝐯1, … , 𝐯𝑝 ∈ ℝ𝑛 , then the set of alle linear combinations of 𝐯1, … , 𝐯𝑝 is denoted by:
𝑆𝑝𝑎𝑛{𝐯1, … , 𝐯𝑝 } and is called the subset of ℝ𝑛 spanned by 𝐯1, … , 𝐯𝑝 .
So 𝑆𝑝𝑎𝑛{𝐯1 , … , 𝐯𝑝} is the collection of vectors that can be written in the form
𝑐1 𝐯1 + 𝑐2 𝐯2 + ⋯ + 𝑐𝑝 𝐯𝑝 with 𝑐1 , … , 𝑐𝑝 ∈ ℝ.
Definition
IF 𝐴 is and 𝑚 × 𝑛 matrix with columns 𝐚1 , … , 𝐚𝑛 and 𝒙 ∈ ℝ𝑛 then the product of 𝐴 and 𝒙 is denoted
by 𝐴𝒙 and is the linear combination of the columns of 𝐴 using the corresponding entries in 𝒙 as
weights.
Intermezzo
The equation 𝐴𝒙 = 𝒃 has a solution if and only if 𝒃 is a linear combination of the columns of 𝐴.
Theorem 1.4
Let 𝐴 be an 𝑚 × 𝑛 matrix. Then the following statements are equivalent:
a) For each 𝒃 ∈ ℝ𝑛 , the equation 𝐴𝒙 = 𝒃 has a solution
b) Each 𝒃 ∈ ℝ𝑛 is a linear combination of the columns of 𝐴
c) 𝑆𝑝𝑎𝑛{𝐚1 , … , 𝐚𝑛 } = ℝ𝑚
d) 𝐴 has a pivot in each row
Theorem 1.5
If 𝐴 is an 𝑚 × 𝑛 matrix and 𝒖, 𝒗 ∈ ℝ𝑛 and 𝑐 is a scalar then:
1) 𝐴(𝐮 + 𝐯) = 𝐴𝐮 + 𝐴𝐯
2) 𝐴(𝑐𝐮) = 𝑐(𝐴𝐮)
Proof
𝑢1 + 𝑣1
1) 𝐴(𝐮 + 𝐯) = [𝐚1 𝐚2 𝐚3 ] [𝑢2 + 𝑣2] = (𝑢1 + 𝑣1 )𝐚1 + (𝑢2 + 𝑣2 ) 𝐚2 + (𝑢3 + 𝑣3)𝐚3 =
𝑢 3 + 𝑣3
( 𝑢1𝐚1 + 𝑢2 𝐚2 + 𝑢3 𝐚3) + (𝑣1 𝐚1 + 𝑣2𝐚2 + 𝑣3𝐚3 ) = 𝐴𝐮 + 𝐴𝐯
2) The way to proof it follows from 1
Definition Homogenous and Nonhomogeneous systems
A linear system is called homogeneous if it can be written in the form 𝐴𝒙 = 𝟎.
A linear system is called nonhomogeneous if it can’t be written in the form 𝐴𝒙 = 𝟎.
A homogeneous system goes trough the origin and a nonhomogeneous system doesn’t.
The homogeneous equation 𝐴𝒙 = 𝟎 always has the trivial solution (𝒙 = 𝟎) and has a nontrivial
solution if and only if the equation has at least one free variable.
We can describe an equation in an implicit way (10𝑥1 − 3𝑥2 − 2𝑥3 = 0) and an explicit way.
An explicit way is called a parametric vector equation which can be written as 𝐱 = 𝑠𝐮 + 𝑡𝐯 (𝑠, 𝑡 ∈ ℝ)
𝑥1 . 3𝑥2 + .2𝑥3 .3 .2
𝑥
𝐱 = [ 2] = [ 𝑥2 ] = 𝑥2 [ 1 ] + 𝑥3 [ 0 ] 𝑥2, 𝑥3 ∈ ℝ
𝑥3 𝑥3 0 1
,Theorem 1.6
Suppose 𝐴𝒙 = 𝒃 is consistent for some given 𝒃 and let 𝒑 be a solution. Then the solution set of
𝐴𝒙 = 𝒃 is the set of all vectors of the form 𝒘 = 𝒑 + 𝒗ℎ where 𝒗ℎ is a solution to 𝐴𝒙 = 𝟎
Definition Linear independence and dependence
An set {𝐯1, … , 𝐯𝑝} ∈ ℝ𝑛 is said to be linear independent if 𝑥1 𝐯1 + 𝑥2 𝐯2 + ⋯ + 𝑥𝑝 𝐯𝑝 = 𝟎
only for 𝑥1 = 𝑥2 = ⋯ = 𝑥𝑝 = 0 (only has the trivial solution)
{𝐯1 , … , 𝐯𝑝 } is said to be linear dependent if ∃𝑐1 , 𝑐2 , … , 𝑐𝑝 not all zero, such that
𝑐1 𝐯1 + 𝑐2 𝐯2 + ⋯ + 𝑐𝑝 𝐯𝑝 = 𝟎
{𝐯1 , … , 𝐯𝑝 } is linear dependent when one of the vectors is a convex combination of the other vectors
in the set.
A set only containing one vector 𝒗 is linear independent only if 𝒗 ≠ 𝟎.
Theorem 1.7 Characterization of Linearly Dependent Sets
An indexed set 𝑆 = {𝐯1, … , 𝐯𝑝} with 𝑝 ≥ 2 is linearly dependent if and only if at least one 𝒗𝑗 ∈ 𝑆 is a
linear combination of the other vectors. If 𝑆 is linearly dependent and 𝒗1 ≠ 𝟎 then some 𝒗𝑗
(with 𝑗 > 1) is a linear combination of the preceding vectors 𝐯1, … , 𝐯𝑗−1
Proof
Let 𝑆 = {𝐯1, … , 𝐯𝑝} ∈ ℝ𝑛 be a linear dependent set and 𝒗𝑗 ∈ S with 𝒗𝑗 a combination of the other
vectors in 𝑆. If 𝒗1 = 𝟎 then it is a trivial combination of the other vectors in 𝑆. If 𝒗1 ≠ 𝟎 ⇒
∃𝑐1 , 𝑐2 , … , 𝑐𝑝 not all zero such that 𝑐1 𝐯1 + 𝑐2 𝐯2 + ⋯ + 𝑐𝑝 𝐯𝑝 = 𝟎.
Let 𝑗 be the largest subscript for which 𝑐𝑗 ≠ 0. If 𝑗 = 1, then 𝑐1 𝐯1 = 𝟎 which is impossible because
𝒗1 ≠ 𝟎 so 𝑗 > 1 and 𝑐1 𝐯1 + ⋯ + 𝑐𝑗 𝐯𝑗 + 0𝐯𝑗+1 + ⋯ + 0𝐯𝑝 = 𝟎 ⇔
𝑐𝑗 𝐯𝑗 = −𝑐1 𝐯1 − ⋯ − 𝑐𝑗 −1 𝐯𝑗−1 ⇔
𝑐 𝑐𝑗−1
𝐯𝑗 = (− 1 ) 𝐯1 + ⋯ + (− ) 𝐯𝑗−1 ⇔
𝑐𝑗 𝑐𝑗
Theorem 1.8
If 𝑆 = {𝐯1, … , 𝐯𝑝} ∈ ℝ𝑛 contains more vectors than there are entries in each vector, so 𝑝 > 𝑛, then 𝑆
is linear dependent
Proof
Let 𝐴 = [𝐯1 ⋯ 𝐯𝑝] then 𝐴 is 𝑛 × 𝑝 . If 𝑝 > 𝑛 then there are more variables then equations so
there must be a free variable. Hence 𝐴𝒙 = 𝟎 has a nontrivial solution, so the columns of 𝐴 are linear
dependent.
Theorem 1.9
If a set 𝑆 = {𝐯1 , … , 𝐯𝑝 } ∈ ℝ𝑛 contains the zero vector, then the set is linearly dependent.
The correspondence from 𝒙 to 𝐴𝒙 (𝒃) can be thought of a function from ℝ𝑛 → ℝ𝑚 .
A transformation (or function or mapping) 𝑇 ∶ ℝ𝑛 → ℝ𝑚 is a transformation of a vector 𝒙 ∈ ℝ𝑛 to a
vector 𝑇(𝒙) ∈ ℝ𝑚 (where 𝑇(𝒙) is the same as the vector 𝒃). We denote the computation 𝑇(𝒙) for
each 𝒙 ∈ ℝ𝑛 , computed by 𝐴𝒙 , by 𝒙 ↦ 𝐴𝒙
,Definition Domain, codomain, range and image
The domain of 𝑇 are the vectors 𝒙 ∈ ℝ𝑛 and the codomain of 𝑇 are the vectors 𝑇(𝒙) ∈ ℝ𝑚 .
𝑇(𝒙) is called the image of 𝒙 and the set of all images 𝑇(𝒙) is called the range of 𝑇.
Definition Linearity of a Transformation
A transformation/ mapping 𝑇 is called linear if:
a) 𝑇(𝐮 + 𝐯) = 𝑇(𝐮) + 𝑇(𝐯) for ∀𝒖, 𝒗 ∈ ℝ𝑛 (ℝ𝑛 being the domain of 𝑇)
b) 𝑇(𝑐𝐮) = 𝑐𝑇(𝐮) for ∀𝑐 ∈ ℝ and ∀𝒖 ∈ ℝ𝑛
In other words: Linear transformations preserve the operations of vector addition and scalar
multiplication.
If 𝑇 is a linear transformation it follows that 𝑇 (𝟎) = 𝟎 and 𝑇(𝑐𝐮 + 𝑑𝐯) = 𝑐𝑇(𝐮) + 𝑑𝑇(𝐯)
Definition Dilation and contraction
Given a scalar 𝑟, define 𝑇 ∶ ℝ𝑛 → ℝ𝑚 by 𝑇(𝒙) = 𝑟𝒙. Then 𝑇 is called a contraction if 0 ≤ 𝑟 ≤ 1
and is called a dilation if 𝑟 > 1.
Definition Onto and one-to-one
A mapping 𝑇 ∶ ℝ𝑛 → ℝ𝑚 is said to be onto (surjective) ℝ𝑚 if each 𝒃 ∈ ℝ𝑚 is the image of at least
one 𝒙 ∈ ℝ𝑛 .
𝑇 is said to be one-to-one (injective) if each 𝒃 ∈ ℝ𝑚 is the image of at most one 𝒙 ∈ ℝ𝑛
That means that 𝑇 is onto ℝ𝑚 when the range of 𝑇 is all of the codomain ℝ𝑚 . That means that if for
𝒃 ∈ 𝑐𝑜𝑑𝑜𝑚𝑎𝑖𝑛 ℝ𝑚 that if ∄𝒙 ∈ ℝ𝑛 such that 𝑇 (𝒙) = 𝒃, then 𝑇 is not onto ℝ𝑚 .
That means that if ∃𝒃 ∈ 𝑐𝑜𝑑𝑜𝑚𝑎𝑖𝑛 ℝ𝑚 such that there are more 𝒙 ∈ ℝ𝑛 for which 𝑇(𝒙) = 𝒃 then 𝑇
is not one-to-one.
Theorem 1.11
Let 𝑇 ∶ ℝ𝑛 → ℝ𝑚 be a linear transformation. Then T is one-to-one if and only if the equation
𝑇(𝒙) = 𝟎 only has the trivial solution.
Proof
Since 𝑇 is linear it follows that 𝑇(𝟎) = 𝟎. If 𝑇 is one-to-one then the equation 𝑇(𝒙) = 𝟎 has at most
one solution and hence only solution the trivial solution, which means it has a unique solution due to
the linear independence of the columns of it’s standard matrix.
If 𝑇 is not one-to-one, then ∃𝒖, 𝒗 ∈ ℝ𝑛 with 𝒖 ≠ 𝒗, such that 𝑇(𝒖) = 𝑇(𝒗) = 𝒃 , then also because
𝑇 is linear: 𝑇(𝒖 − 𝒗) = 𝑇 (𝒖) − 𝑇(𝒗) = 𝒃 − 𝒃 = 𝟎
Theorem 1.12
Let 𝑇 ∶ ℝ𝑛 → ℝ𝑚 be a linear transformation and let 𝐴 be the standardmatrix for 𝑇, then:
a) 𝑇 maps ℝ𝑛 onto ℝ𝑚 if and only if the columns of 𝐴 span ℝ𝑚
b) 𝑇 is one-to-one if and only if the columns of 𝐴 are linear independent.
, Chapter 2 Matrix Algebra
The diagonal entries in an 𝑚 × 𝑛 matrix 𝐴 = [𝑎𝑖𝑗 ] are 𝑎11 , 𝑎22 , 𝑎33 , … and they form the main
diagonal of 𝐴. A diagonal matrix is a square matrix whose nondiagonal entries are zero.
We say two matrices are equal if they have the same size and their entries are the same.
The vector 𝐴(𝐵𝒙) can be written as 𝐴(𝐵𝐱) = [𝐴𝐛1 𝐴𝐛2 ⋯ 𝐴𝐛𝑝]𝐱
This leads to the useful fact that each columns of 𝐴𝐵 is a linear combination of the columns of 𝐴
using weights from the corresponding column of 𝐵.
Row-Column rule for computing product of two matrices
Let us define the product 𝐴𝐵 and let 𝑖 denote the column and 𝑗 the row.
(𝐴𝐵)𝑖 = 𝐴 ∙ 𝐵𝑖
(𝐴𝐵 ) 𝑗 = 𝐴𝑗 ∙ 𝐵
(𝐴𝐵)𝑖𝑗 = 𝐴𝑗 ∙ 𝐵𝑖
Theorem 2.2
a) 𝐴(𝐵𝐶) = (𝐴𝐵 ) 𝐶 (associative law of multiplication)
b) 𝐴(𝐵 + 𝐶) = 𝐴𝐵 + 𝐴𝐶 (left distributive law)
c) (𝐴 + 𝐵 )𝐶 = 𝐴𝐶 + 𝐵𝐶 (right distributive law)
d) 𝑟 (𝐴𝐵 ) = (𝑟𝐴) 𝐵 = 𝐴(𝑟𝐵) for any scalar 𝑟
Intermezzo
If 𝐴𝐵 = 𝐵𝐴 , we say that 𝐴 and 𝐵 commute with one another.
Warnings:
1) In general 𝐴𝐵 ≠ 𝐵𝐴
2) If a product 𝐴𝐵 = 0 (the zeromatrix) we cannot conclude in general that either 𝐴 = 0 or 𝐵 = 0
(zeromatrices)
Definition Transpose
𝐴 ∈ ℝ𝑚×𝑛 then the transpose of 𝐴, is 𝐴𝑇 ∈ ℝ𝑛×𝑚 whose columns are the corresponding rows of 𝐴
Theorem 2.3 Arithmetic rules for Transpose matrix algebra
Let 𝐴 and 𝐵 denote matrices with appropriate sizes for the following sums and products
a) (𝐴𝑇 )𝑇 = 𝐴
b) (𝐴 + 𝐵)𝑇 = 𝐴𝑇 + 𝐵 𝑇
c) For any scalar 𝑟, (𝑟𝐴)𝑇 = 𝑟𝐴𝑇
d) (𝐴𝐵)𝑇 = 𝐵 𝑇 𝐴𝑇
Definition Invertible matrix
Let 𝐴 ∈ ℝ𝑛×𝑛 , then 𝐴 is said to be invertible if ∃𝐶 ∈ ℝ𝑛×𝑛 such that 𝐶𝐴 = 𝐴𝐶 = 𝐼
In this case 𝐶 is called an inverse of 𝐴.
In fact 𝐶 is uniquely determined by 𝐴 because if there would be another matrix 𝐵 such that 𝐵 is an
inverse of 𝐴 then we would have 𝐵 = 𝐵𝐼 = 𝐵𝐴𝐶 = 𝐼𝐶 = 𝐶.
A matrix that is not invertible is called a singular matrix. A matrix that is invertible is called a
nonsingular matrix.
Chapter 1 Linear Equations in Linear Algebra
A linear equation with variables 𝑥1 , … , 𝑥𝑛 is an equation that can be written in the form
𝑎1 𝑥1 + 𝑎2 𝑥2 + ⋯ + 𝑎𝑛 𝑥𝑛 = 𝑏
A system is said to be consistent if it has either one or infinitely many solutions. And is inconsistent if
it has no solution.
Definition Row Equivalent
Two matrices are called row equivalent if there is a sequence of elementary row operations that
transforms one matrix into the other.
Intermezzo
IF the augmented matrices of two linear systems are row equivalent, then the two systems have the
same solution set.
Definition Echelon form and reduced echelon form
A rectangular matrix is in echelon form if it has the following three properties:
1. All nonzero rows are above any zero-rows
2. Each leading entry of a row is in a column to the right of the leading entry of the row above it
3. All entries in a column below a leading entry are zeros
A rectangular matrix is in reduced echelon form if it has the following additional properties:
4. The leading entry in every nonzero row is 1
5. Each leading 1 is the only nonzero entry in its column.
Theorem 1.1 Uniqueness of the Reduced Echelon Form
Each matrix is row equivalent to one and only one reduced echelon matrix.
Definition Pivot position and column
A pivot position in a matrix 𝐴 is a location in 𝐴 that corresponds to a leading 1 in the reduced
echelon form. The pivot column is the column of 𝐴 in which the pivot position is in.
Theorem 1.2 Existence and Uniqueness Theorem
A linear system is consistent if and only if the right most column (the 𝒃-column) of the augmented
matrix is not a pivot column.
If a linear system is consistent, then the solution set either contains an unique solution or infinitely
many.
Intermezzo
A vector equation 𝑥1 𝐚1 + 𝑥2 𝐚2 + ⋯ + 𝑥𝑛 𝐚𝑛 = 𝐛 has the same solution set as the linear system
whose augmented matrix is [𝐚1 𝐚2 ⋯ 𝐚𝑛 𝐛]
,Definition Spanned
If vectors 𝐯1, … , 𝐯𝑝 ∈ ℝ𝑛 , then the set of alle linear combinations of 𝐯1, … , 𝐯𝑝 is denoted by:
𝑆𝑝𝑎𝑛{𝐯1, … , 𝐯𝑝 } and is called the subset of ℝ𝑛 spanned by 𝐯1, … , 𝐯𝑝 .
So 𝑆𝑝𝑎𝑛{𝐯1 , … , 𝐯𝑝} is the collection of vectors that can be written in the form
𝑐1 𝐯1 + 𝑐2 𝐯2 + ⋯ + 𝑐𝑝 𝐯𝑝 with 𝑐1 , … , 𝑐𝑝 ∈ ℝ.
Definition
IF 𝐴 is and 𝑚 × 𝑛 matrix with columns 𝐚1 , … , 𝐚𝑛 and 𝒙 ∈ ℝ𝑛 then the product of 𝐴 and 𝒙 is denoted
by 𝐴𝒙 and is the linear combination of the columns of 𝐴 using the corresponding entries in 𝒙 as
weights.
Intermezzo
The equation 𝐴𝒙 = 𝒃 has a solution if and only if 𝒃 is a linear combination of the columns of 𝐴.
Theorem 1.4
Let 𝐴 be an 𝑚 × 𝑛 matrix. Then the following statements are equivalent:
a) For each 𝒃 ∈ ℝ𝑛 , the equation 𝐴𝒙 = 𝒃 has a solution
b) Each 𝒃 ∈ ℝ𝑛 is a linear combination of the columns of 𝐴
c) 𝑆𝑝𝑎𝑛{𝐚1 , … , 𝐚𝑛 } = ℝ𝑚
d) 𝐴 has a pivot in each row
Theorem 1.5
If 𝐴 is an 𝑚 × 𝑛 matrix and 𝒖, 𝒗 ∈ ℝ𝑛 and 𝑐 is a scalar then:
1) 𝐴(𝐮 + 𝐯) = 𝐴𝐮 + 𝐴𝐯
2) 𝐴(𝑐𝐮) = 𝑐(𝐴𝐮)
Proof
𝑢1 + 𝑣1
1) 𝐴(𝐮 + 𝐯) = [𝐚1 𝐚2 𝐚3 ] [𝑢2 + 𝑣2] = (𝑢1 + 𝑣1 )𝐚1 + (𝑢2 + 𝑣2 ) 𝐚2 + (𝑢3 + 𝑣3)𝐚3 =
𝑢 3 + 𝑣3
( 𝑢1𝐚1 + 𝑢2 𝐚2 + 𝑢3 𝐚3) + (𝑣1 𝐚1 + 𝑣2𝐚2 + 𝑣3𝐚3 ) = 𝐴𝐮 + 𝐴𝐯
2) The way to proof it follows from 1
Definition Homogenous and Nonhomogeneous systems
A linear system is called homogeneous if it can be written in the form 𝐴𝒙 = 𝟎.
A linear system is called nonhomogeneous if it can’t be written in the form 𝐴𝒙 = 𝟎.
A homogeneous system goes trough the origin and a nonhomogeneous system doesn’t.
The homogeneous equation 𝐴𝒙 = 𝟎 always has the trivial solution (𝒙 = 𝟎) and has a nontrivial
solution if and only if the equation has at least one free variable.
We can describe an equation in an implicit way (10𝑥1 − 3𝑥2 − 2𝑥3 = 0) and an explicit way.
An explicit way is called a parametric vector equation which can be written as 𝐱 = 𝑠𝐮 + 𝑡𝐯 (𝑠, 𝑡 ∈ ℝ)
𝑥1 . 3𝑥2 + .2𝑥3 .3 .2
𝑥
𝐱 = [ 2] = [ 𝑥2 ] = 𝑥2 [ 1 ] + 𝑥3 [ 0 ] 𝑥2, 𝑥3 ∈ ℝ
𝑥3 𝑥3 0 1
,Theorem 1.6
Suppose 𝐴𝒙 = 𝒃 is consistent for some given 𝒃 and let 𝒑 be a solution. Then the solution set of
𝐴𝒙 = 𝒃 is the set of all vectors of the form 𝒘 = 𝒑 + 𝒗ℎ where 𝒗ℎ is a solution to 𝐴𝒙 = 𝟎
Definition Linear independence and dependence
An set {𝐯1, … , 𝐯𝑝} ∈ ℝ𝑛 is said to be linear independent if 𝑥1 𝐯1 + 𝑥2 𝐯2 + ⋯ + 𝑥𝑝 𝐯𝑝 = 𝟎
only for 𝑥1 = 𝑥2 = ⋯ = 𝑥𝑝 = 0 (only has the trivial solution)
{𝐯1 , … , 𝐯𝑝 } is said to be linear dependent if ∃𝑐1 , 𝑐2 , … , 𝑐𝑝 not all zero, such that
𝑐1 𝐯1 + 𝑐2 𝐯2 + ⋯ + 𝑐𝑝 𝐯𝑝 = 𝟎
{𝐯1 , … , 𝐯𝑝 } is linear dependent when one of the vectors is a convex combination of the other vectors
in the set.
A set only containing one vector 𝒗 is linear independent only if 𝒗 ≠ 𝟎.
Theorem 1.7 Characterization of Linearly Dependent Sets
An indexed set 𝑆 = {𝐯1, … , 𝐯𝑝} with 𝑝 ≥ 2 is linearly dependent if and only if at least one 𝒗𝑗 ∈ 𝑆 is a
linear combination of the other vectors. If 𝑆 is linearly dependent and 𝒗1 ≠ 𝟎 then some 𝒗𝑗
(with 𝑗 > 1) is a linear combination of the preceding vectors 𝐯1, … , 𝐯𝑗−1
Proof
Let 𝑆 = {𝐯1, … , 𝐯𝑝} ∈ ℝ𝑛 be a linear dependent set and 𝒗𝑗 ∈ S with 𝒗𝑗 a combination of the other
vectors in 𝑆. If 𝒗1 = 𝟎 then it is a trivial combination of the other vectors in 𝑆. If 𝒗1 ≠ 𝟎 ⇒
∃𝑐1 , 𝑐2 , … , 𝑐𝑝 not all zero such that 𝑐1 𝐯1 + 𝑐2 𝐯2 + ⋯ + 𝑐𝑝 𝐯𝑝 = 𝟎.
Let 𝑗 be the largest subscript for which 𝑐𝑗 ≠ 0. If 𝑗 = 1, then 𝑐1 𝐯1 = 𝟎 which is impossible because
𝒗1 ≠ 𝟎 so 𝑗 > 1 and 𝑐1 𝐯1 + ⋯ + 𝑐𝑗 𝐯𝑗 + 0𝐯𝑗+1 + ⋯ + 0𝐯𝑝 = 𝟎 ⇔
𝑐𝑗 𝐯𝑗 = −𝑐1 𝐯1 − ⋯ − 𝑐𝑗 −1 𝐯𝑗−1 ⇔
𝑐 𝑐𝑗−1
𝐯𝑗 = (− 1 ) 𝐯1 + ⋯ + (− ) 𝐯𝑗−1 ⇔
𝑐𝑗 𝑐𝑗
Theorem 1.8
If 𝑆 = {𝐯1, … , 𝐯𝑝} ∈ ℝ𝑛 contains more vectors than there are entries in each vector, so 𝑝 > 𝑛, then 𝑆
is linear dependent
Proof
Let 𝐴 = [𝐯1 ⋯ 𝐯𝑝] then 𝐴 is 𝑛 × 𝑝 . If 𝑝 > 𝑛 then there are more variables then equations so
there must be a free variable. Hence 𝐴𝒙 = 𝟎 has a nontrivial solution, so the columns of 𝐴 are linear
dependent.
Theorem 1.9
If a set 𝑆 = {𝐯1 , … , 𝐯𝑝 } ∈ ℝ𝑛 contains the zero vector, then the set is linearly dependent.
The correspondence from 𝒙 to 𝐴𝒙 (𝒃) can be thought of a function from ℝ𝑛 → ℝ𝑚 .
A transformation (or function or mapping) 𝑇 ∶ ℝ𝑛 → ℝ𝑚 is a transformation of a vector 𝒙 ∈ ℝ𝑛 to a
vector 𝑇(𝒙) ∈ ℝ𝑚 (where 𝑇(𝒙) is the same as the vector 𝒃). We denote the computation 𝑇(𝒙) for
each 𝒙 ∈ ℝ𝑛 , computed by 𝐴𝒙 , by 𝒙 ↦ 𝐴𝒙
,Definition Domain, codomain, range and image
The domain of 𝑇 are the vectors 𝒙 ∈ ℝ𝑛 and the codomain of 𝑇 are the vectors 𝑇(𝒙) ∈ ℝ𝑚 .
𝑇(𝒙) is called the image of 𝒙 and the set of all images 𝑇(𝒙) is called the range of 𝑇.
Definition Linearity of a Transformation
A transformation/ mapping 𝑇 is called linear if:
a) 𝑇(𝐮 + 𝐯) = 𝑇(𝐮) + 𝑇(𝐯) for ∀𝒖, 𝒗 ∈ ℝ𝑛 (ℝ𝑛 being the domain of 𝑇)
b) 𝑇(𝑐𝐮) = 𝑐𝑇(𝐮) for ∀𝑐 ∈ ℝ and ∀𝒖 ∈ ℝ𝑛
In other words: Linear transformations preserve the operations of vector addition and scalar
multiplication.
If 𝑇 is a linear transformation it follows that 𝑇 (𝟎) = 𝟎 and 𝑇(𝑐𝐮 + 𝑑𝐯) = 𝑐𝑇(𝐮) + 𝑑𝑇(𝐯)
Definition Dilation and contraction
Given a scalar 𝑟, define 𝑇 ∶ ℝ𝑛 → ℝ𝑚 by 𝑇(𝒙) = 𝑟𝒙. Then 𝑇 is called a contraction if 0 ≤ 𝑟 ≤ 1
and is called a dilation if 𝑟 > 1.
Definition Onto and one-to-one
A mapping 𝑇 ∶ ℝ𝑛 → ℝ𝑚 is said to be onto (surjective) ℝ𝑚 if each 𝒃 ∈ ℝ𝑚 is the image of at least
one 𝒙 ∈ ℝ𝑛 .
𝑇 is said to be one-to-one (injective) if each 𝒃 ∈ ℝ𝑚 is the image of at most one 𝒙 ∈ ℝ𝑛
That means that 𝑇 is onto ℝ𝑚 when the range of 𝑇 is all of the codomain ℝ𝑚 . That means that if for
𝒃 ∈ 𝑐𝑜𝑑𝑜𝑚𝑎𝑖𝑛 ℝ𝑚 that if ∄𝒙 ∈ ℝ𝑛 such that 𝑇 (𝒙) = 𝒃, then 𝑇 is not onto ℝ𝑚 .
That means that if ∃𝒃 ∈ 𝑐𝑜𝑑𝑜𝑚𝑎𝑖𝑛 ℝ𝑚 such that there are more 𝒙 ∈ ℝ𝑛 for which 𝑇(𝒙) = 𝒃 then 𝑇
is not one-to-one.
Theorem 1.11
Let 𝑇 ∶ ℝ𝑛 → ℝ𝑚 be a linear transformation. Then T is one-to-one if and only if the equation
𝑇(𝒙) = 𝟎 only has the trivial solution.
Proof
Since 𝑇 is linear it follows that 𝑇(𝟎) = 𝟎. If 𝑇 is one-to-one then the equation 𝑇(𝒙) = 𝟎 has at most
one solution and hence only solution the trivial solution, which means it has a unique solution due to
the linear independence of the columns of it’s standard matrix.
If 𝑇 is not one-to-one, then ∃𝒖, 𝒗 ∈ ℝ𝑛 with 𝒖 ≠ 𝒗, such that 𝑇(𝒖) = 𝑇(𝒗) = 𝒃 , then also because
𝑇 is linear: 𝑇(𝒖 − 𝒗) = 𝑇 (𝒖) − 𝑇(𝒗) = 𝒃 − 𝒃 = 𝟎
Theorem 1.12
Let 𝑇 ∶ ℝ𝑛 → ℝ𝑚 be a linear transformation and let 𝐴 be the standardmatrix for 𝑇, then:
a) 𝑇 maps ℝ𝑛 onto ℝ𝑚 if and only if the columns of 𝐴 span ℝ𝑚
b) 𝑇 is one-to-one if and only if the columns of 𝐴 are linear independent.
, Chapter 2 Matrix Algebra
The diagonal entries in an 𝑚 × 𝑛 matrix 𝐴 = [𝑎𝑖𝑗 ] are 𝑎11 , 𝑎22 , 𝑎33 , … and they form the main
diagonal of 𝐴. A diagonal matrix is a square matrix whose nondiagonal entries are zero.
We say two matrices are equal if they have the same size and their entries are the same.
The vector 𝐴(𝐵𝒙) can be written as 𝐴(𝐵𝐱) = [𝐴𝐛1 𝐴𝐛2 ⋯ 𝐴𝐛𝑝]𝐱
This leads to the useful fact that each columns of 𝐴𝐵 is a linear combination of the columns of 𝐴
using weights from the corresponding column of 𝐵.
Row-Column rule for computing product of two matrices
Let us define the product 𝐴𝐵 and let 𝑖 denote the column and 𝑗 the row.
(𝐴𝐵)𝑖 = 𝐴 ∙ 𝐵𝑖
(𝐴𝐵 ) 𝑗 = 𝐴𝑗 ∙ 𝐵
(𝐴𝐵)𝑖𝑗 = 𝐴𝑗 ∙ 𝐵𝑖
Theorem 2.2
a) 𝐴(𝐵𝐶) = (𝐴𝐵 ) 𝐶 (associative law of multiplication)
b) 𝐴(𝐵 + 𝐶) = 𝐴𝐵 + 𝐴𝐶 (left distributive law)
c) (𝐴 + 𝐵 )𝐶 = 𝐴𝐶 + 𝐵𝐶 (right distributive law)
d) 𝑟 (𝐴𝐵 ) = (𝑟𝐴) 𝐵 = 𝐴(𝑟𝐵) for any scalar 𝑟
Intermezzo
If 𝐴𝐵 = 𝐵𝐴 , we say that 𝐴 and 𝐵 commute with one another.
Warnings:
1) In general 𝐴𝐵 ≠ 𝐵𝐴
2) If a product 𝐴𝐵 = 0 (the zeromatrix) we cannot conclude in general that either 𝐴 = 0 or 𝐵 = 0
(zeromatrices)
Definition Transpose
𝐴 ∈ ℝ𝑚×𝑛 then the transpose of 𝐴, is 𝐴𝑇 ∈ ℝ𝑛×𝑚 whose columns are the corresponding rows of 𝐴
Theorem 2.3 Arithmetic rules for Transpose matrix algebra
Let 𝐴 and 𝐵 denote matrices with appropriate sizes for the following sums and products
a) (𝐴𝑇 )𝑇 = 𝐴
b) (𝐴 + 𝐵)𝑇 = 𝐴𝑇 + 𝐵 𝑇
c) For any scalar 𝑟, (𝑟𝐴)𝑇 = 𝑟𝐴𝑇
d) (𝐴𝐵)𝑇 = 𝐵 𝑇 𝐴𝑇
Definition Invertible matrix
Let 𝐴 ∈ ℝ𝑛×𝑛 , then 𝐴 is said to be invertible if ∃𝐶 ∈ ℝ𝑛×𝑛 such that 𝐶𝐴 = 𝐴𝐶 = 𝐼
In this case 𝐶 is called an inverse of 𝐴.
In fact 𝐶 is uniquely determined by 𝐴 because if there would be another matrix 𝐵 such that 𝐵 is an
inverse of 𝐴 then we would have 𝐵 = 𝐵𝐼 = 𝐵𝐴𝐶 = 𝐼𝐶 = 𝐶.
A matrix that is not invertible is called a singular matrix. A matrix that is invertible is called a
nonsingular matrix.
