MATRIX ALGEBRA REVIEW
(
PRELIMINARIES
A matrix is a way of organizing information.
It is a rectangular array of elements arranged in rows and columns. For example, the following matrix
A has m rows and n columns.
⎡ a11 a12 a13 ... a1n ⎤
⎢ ⎥
⎢ a21 a22 a 23 ... a 2 n ⎥
A = ⎢ a31 a32 a 33 ... a3 n ⎥
⎢ ⎥
⎢ M M M O M ⎥
⎢⎣a m1 a m2 a m3 L amn ⎥⎦
All elements can be identified by a typical element a ij , where i=1,2,…,m denotes rows and j=1,2,…,n
denotes columns.
A matrix is of order (or dimension) m by n (also denoted as (m x n)).
A matrix that has a single column is called a column vector.
A matrix that has a single row is called a row vector.
TRANSPOSE
The transpose of a matrix or vector is formed by interchanging the rows and the columns. A matrix of
order (m x n) becomes of order (n x m) when transposed.
For example, if a (2 x 3) matrix is defined by
⎡ a11 a12 a13 ⎤
A=⎢ ⎥
⎣ a21 a22 a23 ⎦
Then the transpose of A, denoted by A’, is now (3 x 2)
⎡ a11 a 21 ⎤
A′ = ⎢⎢ a12 a22 ⎥⎥
⎢⎣ a13 a 23 ⎥⎦
• ( A′)′ = A
• (kA)′ = kA′ , where k is a scalar.
,SYMMETRIC M ATRIX
When A′ = A , the matrix is called symmetric. That is, a symmetric matrix is a square matrix, in that it
has the same number of rows as it has columns, and the off-diagonal elements are symmetric (i.e.
a ij = a ji for all i and j ).
For example,
⎡ 4 5 − 3⎤
A = ⎢⎢ 5 7 2 ⎥⎥
⎢⎣ − 3 2 10 ⎥⎦
A special case is the identity matrix, which has 1’s on the diagonal positions and 0’s on the off-
diagonal positions.
⎡1 0 L 0⎤
⎢0 1 L 0⎥⎥
I =⎢
⎢M M O M⎥
⎢ ⎥
⎣0 0 L 1⎦
The identity matrix is a diagonal matrix, which can be denoted by diag ( a1 , a2 ,..., an ) , where a i is the
ith element on the diagonal position and zeros occur elsewhere. So, we can write the identity matrix as
I = diag(1,1,...,1) .
ADDITION AND S UBTRACTION
Matrices can be added and subtracted as long as they are of the same dimension. The addition of
matrix A and matrix B is the addition of the corresponding elements of A and B. So, C = A + B
implies that cij = a ij + bij for all i and j.
For example, if
⎡ 2 − 3⎤ ⎡0 6 ⎤
A=⎢ ⎥ B =⎢ ⎥
⎣ 6 10 ⎦ ⎣5 − 8⎦
Then
⎡ 2 3⎤
C=⎢ ⎥
⎣11 2⎦
• A± B = B ± A
• ( A ± B) ± C = A ± (B ± C )
• ( A ± B)′ = A′ ± B′
2
, M ULTIPLICATION
If k is a scalar and A is a matrix, then the product of k times A is called scalar multiplication. The
product is k times each element of A. That is, if B = kA , then bij = kaij for all i and j.
In the case of multiplying two matrices, such as C = AB , where neither A nor B are scalars, it must be
the case that
the number of columns of A = the number of rows of B
So, if A is of dimension (m x p) and B of dimension (p x n), then the product, C, will be of order (m x
n) whose ijth element is defined as
p
cij = ∑ a ik bkj
k =1
In words, the ijth element of the product matrix is found by multiplying the elements of the ith row of A,
the first matrix, by the corresponding elements of the jth column of B, the second matrix, and summing
the resulting product. For this to hold, the number of columns in the first matrix must equal the
number of rows in the second.
For example,
⎡ 6 8 ⎤ ⎡3 − 8 1⎤
F = AD = ⎢ ⎥⎢ ⎥
⎣− 2 4⎦ ⎣9 2 5⎦
⎡ 6* 3 + 8* 9 6 * (−8) + 8 * 2 6 *1 + 8 * 5 ⎤
=⎢ ⎥
⎣( −2) * 3 + 4 * 9 ( −2) * ( −8) + 4 * 2 (−2) *1 + 4 * 5⎦
⎡90 − 32 46⎤
=⎢ ⎥
⎣30 24 18 ⎦
• A (m x 1) column vector multiplied by a (1 x n) row vector becomes an (m x n) matrix.
• A (1 x m) row vector multiplied by a (m x 1) column vector becomes a scalar.
• In general, AB ≠ BA .
• But, kA = Ak if k is a scalar and A is a matrix.
• And, AI = IA if A is a matrix and I is the identity matrix and conformable for multiplication.
The product of a row vector and a column vector of the same dimension is called the inner product
(also called the dot product), its value is the sum of products of the components of the vectors. For
example, if j is a (T x 1) vector with elements 1, then the inner product, j’j, is equal to a constant T.
Note: two vectors are orthogonal if their inner product is zero.
• A( B + C) = AB + AC .
• ( A + B)C = AC + BC .
3
(
PRELIMINARIES
A matrix is a way of organizing information.
It is a rectangular array of elements arranged in rows and columns. For example, the following matrix
A has m rows and n columns.
⎡ a11 a12 a13 ... a1n ⎤
⎢ ⎥
⎢ a21 a22 a 23 ... a 2 n ⎥
A = ⎢ a31 a32 a 33 ... a3 n ⎥
⎢ ⎥
⎢ M M M O M ⎥
⎢⎣a m1 a m2 a m3 L amn ⎥⎦
All elements can be identified by a typical element a ij , where i=1,2,…,m denotes rows and j=1,2,…,n
denotes columns.
A matrix is of order (or dimension) m by n (also denoted as (m x n)).
A matrix that has a single column is called a column vector.
A matrix that has a single row is called a row vector.
TRANSPOSE
The transpose of a matrix or vector is formed by interchanging the rows and the columns. A matrix of
order (m x n) becomes of order (n x m) when transposed.
For example, if a (2 x 3) matrix is defined by
⎡ a11 a12 a13 ⎤
A=⎢ ⎥
⎣ a21 a22 a23 ⎦
Then the transpose of A, denoted by A’, is now (3 x 2)
⎡ a11 a 21 ⎤
A′ = ⎢⎢ a12 a22 ⎥⎥
⎢⎣ a13 a 23 ⎥⎦
• ( A′)′ = A
• (kA)′ = kA′ , where k is a scalar.
,SYMMETRIC M ATRIX
When A′ = A , the matrix is called symmetric. That is, a symmetric matrix is a square matrix, in that it
has the same number of rows as it has columns, and the off-diagonal elements are symmetric (i.e.
a ij = a ji for all i and j ).
For example,
⎡ 4 5 − 3⎤
A = ⎢⎢ 5 7 2 ⎥⎥
⎢⎣ − 3 2 10 ⎥⎦
A special case is the identity matrix, which has 1’s on the diagonal positions and 0’s on the off-
diagonal positions.
⎡1 0 L 0⎤
⎢0 1 L 0⎥⎥
I =⎢
⎢M M O M⎥
⎢ ⎥
⎣0 0 L 1⎦
The identity matrix is a diagonal matrix, which can be denoted by diag ( a1 , a2 ,..., an ) , where a i is the
ith element on the diagonal position and zeros occur elsewhere. So, we can write the identity matrix as
I = diag(1,1,...,1) .
ADDITION AND S UBTRACTION
Matrices can be added and subtracted as long as they are of the same dimension. The addition of
matrix A and matrix B is the addition of the corresponding elements of A and B. So, C = A + B
implies that cij = a ij + bij for all i and j.
For example, if
⎡ 2 − 3⎤ ⎡0 6 ⎤
A=⎢ ⎥ B =⎢ ⎥
⎣ 6 10 ⎦ ⎣5 − 8⎦
Then
⎡ 2 3⎤
C=⎢ ⎥
⎣11 2⎦
• A± B = B ± A
• ( A ± B) ± C = A ± (B ± C )
• ( A ± B)′ = A′ ± B′
2
, M ULTIPLICATION
If k is a scalar and A is a matrix, then the product of k times A is called scalar multiplication. The
product is k times each element of A. That is, if B = kA , then bij = kaij for all i and j.
In the case of multiplying two matrices, such as C = AB , where neither A nor B are scalars, it must be
the case that
the number of columns of A = the number of rows of B
So, if A is of dimension (m x p) and B of dimension (p x n), then the product, C, will be of order (m x
n) whose ijth element is defined as
p
cij = ∑ a ik bkj
k =1
In words, the ijth element of the product matrix is found by multiplying the elements of the ith row of A,
the first matrix, by the corresponding elements of the jth column of B, the second matrix, and summing
the resulting product. For this to hold, the number of columns in the first matrix must equal the
number of rows in the second.
For example,
⎡ 6 8 ⎤ ⎡3 − 8 1⎤
F = AD = ⎢ ⎥⎢ ⎥
⎣− 2 4⎦ ⎣9 2 5⎦
⎡ 6* 3 + 8* 9 6 * (−8) + 8 * 2 6 *1 + 8 * 5 ⎤
=⎢ ⎥
⎣( −2) * 3 + 4 * 9 ( −2) * ( −8) + 4 * 2 (−2) *1 + 4 * 5⎦
⎡90 − 32 46⎤
=⎢ ⎥
⎣30 24 18 ⎦
• A (m x 1) column vector multiplied by a (1 x n) row vector becomes an (m x n) matrix.
• A (1 x m) row vector multiplied by a (m x 1) column vector becomes a scalar.
• In general, AB ≠ BA .
• But, kA = Ak if k is a scalar and A is a matrix.
• And, AI = IA if A is a matrix and I is the identity matrix and conformable for multiplication.
The product of a row vector and a column vector of the same dimension is called the inner product
(also called the dot product), its value is the sum of products of the components of the vectors. For
example, if j is a (T x 1) vector with elements 1, then the inner product, j’j, is equal to a constant T.
Note: two vectors are orthogonal if their inner product is zero.
• A( B + C) = AB + AC .
• ( A + B)C = AC + BC .
3
