Chapter 2

2.1 Matrix Operations

If A is an m x n matrix - that is, a matrix with m rows and n columns - then the scalar entry in the ith row and the jth column of A is denoted by a_ij and is called the (i, j)-entry of A.

Each column of A is a list of m real numbers, which identifies a vector in R^m. Often, these columns are denoted by a₁, ..., a_n, and matrix A is written as A = [ a₁ a₂ ... a_n ]. Observe that the number a_ij is the ith entry (from the top) of the jth column vector a_j.

The diagonal entries in an m x n matrix A = [ a_ij ] are a₁₁, a₂₂, a₃₃, ..., and they form the main diagonal of A.

A diagonal matrix is a square n x n matrix whose nondiagonal entries are zero.

An m x n matrix whose entries are all zero is a zero matrix and is written as 0.

We say that two matrices are equal if they have the same size (i.e., the same number of rows and the same number of columns) and if their corresponding columns are equal, which amounts to saying that their corresponding entries are equal.

If A and B are m x n matrices, then the sum A + B is the m x n matrix whose columns are the sums of the corresponding columns in A and B. Since vector addition of the columns is done entrywise, each entry in A + B is the sum of the corresponding entries in A and B.
_{Note: The sum A + B is defined only when A and B are the same size.}

If r is a scalar and A is a matrix, then the scalar multiple rA is the matrix whose columns are r times the corresponding columns in A.

Let A, B, and C be matrices of the same size, and let r and s be scalars.
a. A + B = B + A.
b. (A + B) + C = A + (B + C).
c. A + 0 = A.
d. r(A + B) = rA + rB.
e. (r + s)A = rA + sA.
f. r(sA) = (rs)A.

If A is an m x n matrix, and if B is an n x p matrix with columns b₁, ..., b_p, then the product AB is the m x p matrix whose columns are Ab₁, ..., Ab_p. That is, AB = A[ b₁ b₂ ... b_p ] = [ Ab₁ Ab₂ ... Ab_p ].
_{Note: AB has the same number of rows as A and the same number of columns as B.}

Each column of AB is a linear combination of the columns of A using weights from the corresponding column of B.

Row Column Rule for Computing AB
If the product AB is defined, then the entry in row i and column j of AB is the sum of the products of corresponding entries from row i of A and column j of B. If (AB)_ij denotes the (i, j)-entry in AB, and if A is an m x n matrix, then (AB)_ij = a_i1b_1j + a_i2b_2j + ... + a_inb_nj.

Let row_i(A) denote the ith row of a matrix A. Then row_i(AB) = row_i(A) · B.

Let A be an m x n matrix, and let B and C have sizes for which the indicated sums and products are defined.
a. A(BC) = (AB)C. (associative law of multiplication)
b. A(B + C) = AB + AC. (left distributive law)
c. (B + C)A = BA + CA. (right distributive law)
d. r(AB) = (rA)B = A(rB). (for any scalar r)
e. I_mA = A = AI_n. (identity for matrix multiplication)

Warnings

1. In general, AB is not equal to BA.
2. The cancellation laws do not hold for matrix multiplication. That is, if AB = AC, then it is not true in general that B = C.
3. If a product AB is the zero matrix, you cannot conclude in general that either A = 0 or B = 0.

If A is an n x n matrix and if k is a positive integer, then A^k denotes the product of k copies of A.

If A is nonzero and if x is in Rⁿ, then A^kx is the result of left-multiplying x by A repeatedly k times. if k = 0, then A⁰x should be x itself. Thus A⁰ is interpreted as the identity matrix.

Given an m x n matrix A, the transpose of A is the n x m matrix, denoted by A^T, whose columns are formed from the corresponding rows of A.

Let A and B denote matrices whose sizes are appropriate for the following sums and products.
a. (A^T)^T = A.
b. (A + B)^T = A^T + B^T.
c. For any scalar r, (rA)^T = rA^T.
d. (AB)^T = B^TA^T.
_{Note: The transpose of a product of matrices equals the product of their transposes in the reverse order.}

2.2 The Inverse of Matrices

An n x n matrix A is said to be invertible if there is an n x n matrix C such that CA = I and AC = I where I = I_n, the n x n identity matrix. This unique inverse is denoted by A^-1, so that A^-1A = I and AA^-1 = I.

A matrix that is not invertible is sometimes called a singular matrix, and an invertible matrix is called a nonsingular matrix.

Let A be a 2 x 2 matrix with elements a, b, c, and d. The determinant of A, denoted by det A, is the quantity ad - bc. Matrix A is invertible if and only if det A is not equal to 0.

If A is an invertible n x n matrix, then for each b in Rⁿ, the equation Ax = b has the unique solution x = A^-1b.

If A is an invertible matrix, then A^-1 is invertible and (A^-1)^-1 = A.

If A and B are n x n invertible matrices, then so is AB, and the inverse of AB is the product of the inverses of A and B in the reverse order. That is, (AB)^-1 = B^-1A^-1.

If A is an invertible matrix, then so is A^T, and the inverse of A^T is the transpose of A^-1. That is, (A^T)^-1 = (A^-1)^T.

An elementary matrix is one that is obtained by performing a single elementary row operation on an identity matrix.

If an elementary row operation is performed on an m x n matrix A, the resulting matrix can be written as EA, where the m x n matrix E is created by performing the same row operation on I_m.

Each elementary matrix E is invertible. The inverse of E is the elementary matrix of the same type that transforms E back into I.

An n x n matrix A is invertible if and only if A is row equivalent to I_n, and in this case, any sequence of elementary row operations that reduces A to I_n also transforms I_n into A^-1.

2.3 Characterizations of Invertible Matrices

Let A and B be square matrices. If AB = I, then A and B are both invertible, with B = A^-1 and A = B⁰¹.

Let T : Rⁿ -> Rⁿ be a linear transformation and let A be the standard matrix for T. Then T is invertible if and only if A is an invertible matrix. In that case, the linear transformation S given by S(x) = A^-1x is the unique function satisfying the equations S(T(x)) = x for all x in Rⁿ and T(S(x)) = x for all x in Rⁿ.

2.8 Subspaces of Rⁿ

A subspace of Rⁿ is any set H in Rⁿ that has three properties:
a. The zero vector is in H.
b. For each u and v in H, the sum u+v is in H.
c. For each u in H and scalar c, the vector cu is in H.

The column space of a matrix A is the set Col A of all linear combinations of the columns of A.

The null space of a matrix A is the set Nul A of all solutions of the homogeneous equation Ax = 0.

The null space of an m x n matrix A is a subspace of Rⁿ. Equivalently, the set of all solutions of a system Ax = 0 of m homogeneous linear equations in n unknowns is a subspace of Rⁿ.

A basis for a subspace H of Rⁿ is a linearly independent set in H that spans H.

The pivot columns of a matrix A form a basis for the column space of A.

2.9 Dimension and Rank

Suppose the set B = {b₁, ..., b_p} is a basis for a subspace H. For each x in H, the coordinates of x relative to the basis B are weights c₁, ..., c_p such that x = c₁b₁ + ... + c_pb_p, and the vector in R^p [x]_B = [c₁, ..., c_p] is called the coordinate vector of x (relative to B) or the B-coordinate vector of x.

The dimension of a nonzero subspace H, denoted by dim H, is the number of vectors in any basis for H. The dimension of the zero subspace {0} is defined to be zero.

The rank of a matrix A, denoted by rank A, is the dimension of the column space of A.

The Rank Theorem
If a matrix A has n columns, then rank A + dim Nul A = n.

The Basis Theorem
Let H be a p-dimensional subspace of Rⁿ. Any linearly independent set of exactly p elements in H is automatically a basis for H. Also, any set of p elements of H that spans H is automatically a basis for H.