MH1810 Math

Chapter 3: Matrices

Imron Rosyadi

What is a Matrix?

A matrix is a rectangular array of numbers or other mathematical objects. They are fundamental in various fields, from engineering to social sciences.

Why do we use matrices?

• Neat Data Representation: Organize complex data efficiently.
• Information Extraction: Obtain further insights from structured data.
• Wide Applications: Used extensively in modern computing and science.

Matrices are more than just tables of numbers. They are powerful tools for representing and manipulating data, which is crucial in many scientific and engineering disciplines. For example, in computer graphics, matrices are used to transform objects in 3D space, like rotations, scaling, and translations.

Matrix Notation

An \(m \times n\) matrix \(A\) has \(m\) rows and \(n\) columns.

\[ A = \left( \begin{array}{c c c c c c} a _ {1 1} & a _ {1 2} & \dots & a _ {1 j} & \dots & a _ {1 n} \\ a _ {2 1} & a _ {2 2} & \dots & a _ {2 j} & \dots & a _ {2 n} \\ \vdots & \vdots & & \vdots & & \vdots \\ a _ {i 1} & a _ {i 2} & \dots & a _ {i j} & \dots & a _ {i n} \\ \vdots & \vdots & & \vdots & & \\ a _ {m 1} & a _ {m 2} & \dots & a _ {m j} & \dots & a _ {m n} \end{array} \right). \]

• The \((i,j)\)th entry \(a_{ij}\) is found in the \(i\)th row and \(j\)th column.
• Matrices are typically denoted by capital letters (\(A, B, C, \dots\)).
• Their entries are denoted by corresponding lowercase letters (\(a_{ij}, b_{ij}, \dots\)).

This standard notation helps us clearly identify individual elements within a matrix and understand its structure. The indices \(i\) and \(j\) are crucial for pinpointing the exact location of each entry.

Rows and Columns

The structure of a matrix is defined by its rows and columns.

\(i\)th row of \(A\)

(where \(1 \leq i \leq m\))

\[ \left( \begin{array}{c c c c} a _ {i 1} & a _ {i 2} & \dots & a _ {i n} \end{array} \right) \]

\(j\)th column of \(A\)

(where \(1 \leq j \leq n\))

\[ \left( \begin{array}{c} a _ {1 j} \\ a _ {2 j} \\ \vdots \\ a _ {m j} \end{array} \right) \]

Tip

The size of a matrix is given as \(m \times n\), read as “m by n”, where \(m\) is the number of rows and \(n\) is the number of columns.

Understanding rows and columns is fundamental to performing matrix operations. Think of a row as a horizontal list of numbers and a column as a vertical list. The order matters significantly in matrix operations.

Special Types of Matrices

Matrices come in various forms, each with specific properties.

• Row Matrix (Row Vector): One row (\(m=1\)). \[ A = \left[ \begin{array}{l l l l l} a _ {1} & a _ {2} & a _ {3} & \dots & a _ {n} \end{array} \right] \]
• Column Matrix (Column Vector): One column (\(n=1\)). \[ B = \left[ \begin{array}{l} b _ {1} \\ b _ {2} \\ b _ {3} \\ \vdots \\ b _ {m} \end{array} \right] \]

• Zero Matrix (0): All entries are zero. \[ \mathbf{0} = \left[ \begin{array}{c c} 0 & 0 \\ 0 & 0 \end{array} \right] \]
• Square Matrix: Number of rows equals number of columns (\(m=n\)). \[ T = \left( \begin{array}{c c c} a & b & c \\ d & e & f \\ x & y & z \end{array} \right) \]

These special types of matrices simplify certain mathematical problems and operations. For instance, row and column vectors are often used to represent points in space or solutions to systems of equations.

Diagonal Entries & Identity Matrix

In square matrices, specific entries hold special importance.

Diagonal Entries

For an \(n \times n\) square matrix \(A\): \[ A = \left[ \begin{array}{c c c c} a _ {1 1} & a _ {1 2} & \dots & a _ {1 n} \\ a _ {2 1} & a _ {2 2} & \dots & a _ {2 n} \\ \vdots & \vdots & & \vdots \\ a _ {m 1} & a _ {m 2} & \dots & a _ {n n} \end{array} \right]. \] The entries \(a_{11}, a_{22}, \dots, a_{nn}\) are on the main diagonal.

Identity Matrix (\(I_n\))

• An \(n \times n\) square matrix.
• Diagonal entries are 1.
• All other entries are 0.

\[ I _ {2} = \left( \begin{array}{c c} 1 & 0 \\ 0 & 1 \end{array} \right), \quad I _ {3} = \left( \begin{array}{c c c} 1 & 0 & 0 \\ 0 & 1 & 0 \\ 0 & 0 & 1 \end{array} \right). \] The identity matrix acts like the number ‘1’ in scalar multiplication.

The main diagonal is a key feature of square matrices, appearing in many definitions and operations. The identity matrix is particularly important as it serves as a multiplicative identity, similar to how the number 1 works with scalars. We’ll see its significance when we discuss matrix multiplication.

Other Square Matrix Types

Beyond the identity matrix, other structured square matrices exist.

Diagonal Matrix

• All off-diagonal entries are 0.

\[ A = \left( \begin{array}{c c c} a & 0 & 0 \\ 0 & e & 0 \\ 0 & 0 & g \end{array} \right) \]

Upper Triangular Matrix

• All entries below the main diagonal are 0.

\[ A = \left( \begin{array}{c c c} a & b & c \\ 0 & e & f \\ 0 & 0 & g \end{array} \right) \]

Lower Triangular Matrix

• All entries above the main diagonal are 0.

\[ C = \left( \begin{array}{c c} a & 0 \\ c & d \end{array} \right) \]

Note

Scalars refer to real or complex numbers when discussing matrices and vectors.

Triangular matrices are common in solving systems of linear equations and in algorithms for matrix decomposition. Their structured nature simplifies calculations.

Matrix Equality

Two matrices are equal if and only if they have the same size and their corresponding entries are equal.

• Matrices \(A\) and \(B\) are equal if:
  1. They have the same number of rows (\(m\)).
  2. They have the same number of columns (\(n\)).
  3. \(A_{ij} = B_{ij}\) for all \(1 \leq i \leq m\) and \(1 \leq j \leq n\).

Matrix Equality

Solve the following matrix equation for \(a, b, c\) and \(d\):

\[ \left[ \begin{array}{c c} a - b & b + c \\ 3 d + c & 2 a - 4 d \end{array} \right] = \left[ \begin{array}{c c} 8 & 1 \\ 7 & 6 \end{array} \right] \]

This leads to a system of linear equations:

\(a - b = 8\)

\(b + c = 1\)

\(3 d + c = 7\)

\(2 a - 4 d = 6\)

Solving this system gives \(a=5, b=-3, c=4, d=1\).

Matrix equality is a strict definition. Even a single differing entry or a difference in size means the matrices are not equal. This concept is vital for solving matrix equations.

Matrix Addition and Subtraction

Matrices of the same size can be added or subtracted by operating on their corresponding entries.

Addition (\(A+B\))

• If \(A\) and \(B\) are \(m \times n\) matrices: \[ (A + B) _ {i j} = (A) _ {i j} + (B) _ {i j}. \]

Note

Let \(A = \left( \begin{array}{lll}1 & 2 & 3\\ 4 & 5 & 6 \end{array} \right)\) and \(B = \left( \begin{array}{lll}7 & 8 & 9\\ 10 & 11 & 12 \end{array} \right)\).

\[ A + B = \left( \begin{array}{c c c} 1+7 & 2+8 & 3+9 \\ 4+10 & 5+11 & 6+12 \end{array} \right) = \left( \begin{array}{c c c} 8 & 10 & 12 \\ 14 & 16 & 18 \end{array} \right) \]

Subtraction (\(A-B\))

• If \(A\) and \(B\) are \(m \times n\) matrices: \[ (A - B) _ {i j} = (A) _ {i j} - (B) _ {i j}. \]

Note

Using the same \(A\) and \(B\):

\[ A - B = \left( \begin{array}{c c c} 1-7 & 2-8 & 3-9 \\ 4-10 & 5-11 & 6-12 \end{array} \right) = \left( \begin{array}{c c c} -6 & -6 & -6 \\ -6 & -6 & -6 \end{array} \right) \]

Warning

Matrices of different sizes cannot be added or subtracted.

The rule for matrix addition and subtraction is very intuitive: just add or subtract the numbers in the same positions. This is similar to adding or subtracting vectors. The critical constraint is that the matrices must be of the exact same dimensions.

Scalar Multiplication

Multiplying a matrix by a scalar (a real or complex number) involves multiplying each entry of the matrix by that scalar.

• If \(\alpha\) is a scalar and \(A\) is an \(m \times n\) matrix: \[ (\alpha A) _ {i j} = \alpha (A) _ {i j}. \]

Scalar Multiplication

Let \(A = \left( \begin{array}{c c c} 1 & 3 & 5 \\ 7 & 9 & 1 1 \end{array} \right)\). Then:

\[ 2 A = \left( \begin{array}{c c c} 2 & 6 & 1 0 \\ 1 4 & 1 8 & 2 2 \end{array} \right) \] \[ (- 3) A = \left( \begin{array}{c c c} - 3 & - 9 & - 1 5 \\ - 2 1 & - 2 7 & - 3 3 \end{array} \right) \] \[ \frac {1}{3} A = \left( \begin{array}{c c c} \frac {1}{3} & 1 & \frac {5}{3} \\ \frac {7}{3} & 3 & \frac {1 1}{3} \end{array} \right) \]

Scalar multiplication scales the entire matrix. Every element gets multiplied by the same scalar. This is a linear operation and is fundamental to many other matrix operations. Also, note that \(A - B = A + (-1)B\), where \(-B\) is a common notation for \((-1)B\).

Properties of Matrix Arithmetic

Matrix arithmetic, including addition, subtraction, and scalar multiplication, follows familiar rules.

Assuming matrix sizes allow operations:

1. Commutative Law for Addition: \(A + B = B + A\)
2. Associative Law for Addition: \((A + B) + C = A + (B + C)\)
3. Additive Identity: \(A + 0 = 0 + A = A\)
4. Additive Inverse: \(A + (-A) = 0\)
5. Scalar Distribution: \(\alpha (A \pm B) = \alpha A \pm \alpha B\)
6. Scalar Distribution: \((\alpha \pm \beta)A = \alpha A \pm \beta A\)
7. Scalar Associativity: \(\alpha (\beta A) = (\alpha \beta)A\)

Note

These properties are analogous to those of real numbers, reflecting the entry-wise nature of these operations.

These properties are incredibly useful for simplifying matrix expressions and solving matrix equations. They demonstrate that these basic matrix operations behave very much like arithmetic with numbers, as long as the dimensions are compatible.

Matrix Multiplication: The Dot Product Connection

Matrix multiplication is not element-wise. Instead, it’s defined using the dot product of rows and columns.

• If \(A\) is an \(m \times r\) matrix and \(B\) is an \(r \times n\) matrix, their product \(AB\) is an \(m \times n\) matrix.
• The \((i,j)\)th entry of \(AB\) is the dot product of the \(i\)th row of \(A\) and the \(j\)th column of \(B\).

\[ (A B) _ {i j} = A _ {i 1} B _ {1 j} + A _ {i 2} B _ {2 j} + \dots + A _ {i r} B _ {r j} = \sum_ {k = 1} ^ {r} A _ {i k} B _ {k j}. \]

Important

For the product \(AB\) to be defined, the number of columns in \(A\) (\(r\)) must equal the number of rows in \(B\) (\(r\)). The resulting matrix \(AB\) will have dimensions \(m \times n\).

This definition of matrix multiplication might seem complex at first, but it’s designed this way because it naturally arises in many applications, such as representing linear transformations and solving systems of equations. The “inner” dimensions must match, and the “outer” dimensions determine the size of the product.

Visualizing Matrix Multiplication

Let’s break down the process.

\[ A B = \left( \begin{array}{c c c c} A _ {1 1} & A _ {1 2} & \dots & A _ {1 r} \\ A _ {2 1} & A _ {2 2} & \dots & A _ {2 r} \\ \vdots & \vdots & & \vdots \\ A _ {i 1} & A _ {i 2} & \dots & A _ {i r} \\ \vdots & \vdots & & \vdots \\ A _ {m 1} & A _ {m 2} & \dots & A _ {m r} \end{array} \right) \left( \begin{array}{c c c c c c} B _ {1 1} & B _ {1 2} & \dots & B _ {1 j} & \dots & B _ {1 n} \\ B _ {2 1} & B _ {2 2} & \dots & B _ {2 j} & \dots & B _ {2 n} \\ \vdots & \vdots & & \vdots \\ B _ {r 1} & B _ {r 2} & \dots & B _ {r j} & \dots & B _ {r n} \end{array} \right). \]

To find \((AB)_{ij}\):

1. Take the \(i\)th row of matrix \(A\).
2. Take the \(j\)th column of matrix \(B\).
3. Perform the dot product of these two vectors.

Visualizing Matrix Multiplication

Let \(A = \left( \begin{array}{rrr}1 & 2 & -1\\ 3 & 1 & 4 \end{array} \right)\) (\(2 \times 3\)) and \(B = \left( \begin{array}{c} - 2\\ 4\\ 2 \end{array} \right)\) (\(3 \times 1\)).

The product \(AB\) will be a \(2 \times 1\) matrix.

\((AB)_{11} = (1)(-2) + (2)(4) + (-1)(2) = -2 + 8 - 2 = 4\)

\((AB)_{21} = (3)(-2) + (1)(4) + (4)(2) = -6 + 4 + 8 = 6\)

So, \(AB = \left[ \begin{array}{l} 4 \\ 6 \end{array} \right]\).

This example clearly shows how each entry of the product matrix is calculated. It’s crucial to be systematic when performing these calculations, especially with larger matrices. Remember the condition for multiplication: columns of A must equal rows of B. If this condition isn’t met, the product is undefined.

Important Remarks on Matrix Multiplication

Matrix multiplication has distinct properties compared to scalar multiplication.

1. Not Commutative: In general, \(AB \neq BA\).
   • Sometimes \(BA\) might not even be defined if \(AB\) is.
   • Even if both \(AB\) and \(BA\) are defined and are of the same size, they are usually not equal.
2. Zero Product: If \(AB = 0\), it does not necessarily mean \(A = 0\) or \(B = 0\).

Important Remarks on Matrix Multiplication

Find two non-zero \(2 \times 2\) matrices \(A\) and \(B\) such that \(AB = 0\).

Let \(A = \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \end{array} \right]\) and \(B = \left[ \begin{array}{cc} 0 & 0 \\ 0 & 1 \end{array} \right]\). Both are non-zero.

\(AB = \left[ \begin{array}{cc} 1 & 0 \\ 0 & 0 \end{array} \right] \left[ \begin{array}{cc} 0 & 0 \\ 0 & 1 \end{array} \right] = \left[ \begin{array}{cc} (1)(0)+(0)(0) & (1)(0)+(0)(1) \\ (0)(0)+(0)(0) & (0)(0)+(0)(1) \end{array} \right] = \left[ \begin{array}{cc} 0 & 0 \\ 0 & 0 \end{array} \right]\).

These two remarks highlight key differences between matrix algebra and scalar algebra. The non-commutative property means the order of multiplication is critical, and the zero product property means we can’t assume that if a product is zero, one of the factors must be zero. These differences are a common source of error if not kept in mind.

Properties of Matrix Multiplication

Matrix multiplication also has properties related to identity and distribution.

Assuming matrix sizes allow operations:

1. Identity Property:
   • \(AI_{n} = A\) if \(A\) is \(m \times n\).
   • \(I_{m}A = A\) if \(A\) is \(m \times n\).
2. Associative Law for Multiplication: \((AB)C = A(BC)\)
3. Left Distributive Law: \(A(B + C) = AB + AC\)
4. Right Distributive Law: \((A + B)C = AC + BC\)
5. Zero Product Property: \(A0 = 0\) and \(0A = 0\) (where 0 is the zero matrix of appropriate size).

Tip

The identity matrix acts as ‘1’ for matrix multiplication, allowing you to remove it without changing the matrix.

These properties are crucial for manipulating matrix equations. The associativity allows us to group matrix products in any order, which is very useful. The distributive laws are also intuitive. The zero product property is important to distinguish from the scalar zero property, where \(AB=0\) doesn’t imply \(A=0\) or \(B=0\).

Transpose of a Matrix

The transpose of a matrix \(A\), denoted \(A^T\), is obtained by interchanging its rows and columns.

• If \(A\) is an \(m \times n\) matrix, \(A^T\) is an \(n \times m\) matrix.
• The \((i,j)\)th entry of \(A^T\) is the \((j,i)\)th entry of \(A\): \[ (A ^ {T}) _ {i j} = (A) _ {j i} \]

Transpose of a Matrix

Let \(A = \left( \begin{array}{rrr}4 & 3 & 2\\ 1 & 3 & 1 \end{array} \right)\) (\(2 \times 3\)). Then:

\[ A ^ {T} = \left( \begin{array}{c c} 4 & 1 \\ 3 & 3 \\ 2 & 1 \end{array} \right) \] Let \(B = \left( \begin{array}{rrr}1 & 0 & 1\\ -2 & 1 & 0\\ 0 & -3 & -2 \end{array} \right)\) (\(3 \times 3\)). Then:

\[ B ^ {T} = \left( \begin{array}{c c c} 1 & -2 & 0 \\ 0 & 1 & -3 \\ 1 & 0 & -2 \end{array} \right) \]

Taking the transpose effectively ‘flips’ the matrix over its main diagonal. This operation is useful in many areas, including statistics (covariance matrices) and linear algebra (orthogonal matrices).

Properties of the Transpose

The transpose operation has several useful properties:

Assuming matrix sizes allow operations:

1. \((A^T)^T = A\)
2. \((A \pm B)^T = A^T \pm B^T\)
3. \((\alpha A)^T = \alpha (A^T)\) (where \(\alpha\) is a scalar)
4. \((AB)^T = B^T A^T\) (The order of multiplication reverses!)

Important

The property \((AB)^T = B^T A^T\) is crucial and often a point of error. Remember to reverse the order of multiplication when taking the transpose of a product.

These properties are fundamental for algebraic manipulations involving transposes. Property (d) is particularly important and is worth memorizing. It shows that matrix transpose interacts with multiplication in a unique way.

Matrix Inverse

For a square matrix \(A\), its inverse \(A^{-1}\) acts like a reciprocal in scalar arithmetic.

Definition: Let \(A\) be an \(n \times n\) square matrix. If there exists another \(n \times n\) matrix \(B\) such that \(AB = I_n\) and \(BA = I_n\), then:

• \(A\) is invertible (or non-singular).
• \(B\) is the inverse of \(A\), denoted \(A^{-1}\).
• If no such \(B\) exists, \(A\) is not invertible (or singular).

Matrix Inverse

Let \(A = \left( \begin{array}{cc} -1 & -2 \\ 3 & 5 \end{array} \right)\) and \(B = \left( \begin{array}{cc} 5 & 2 \\ -3 & -1 \end{array} \right)\).

Verify:

\(AB = \left( \begin{array}{cc} (-1)(5)+(-2)(-3) & (-1)(2)+(-2)(-1) \\ (3)(5)+(5)(-3) & (3)(2)+(5)(-1) \end{array} \right) = \left( \begin{array}{cc} -5+6 & -2+2 \\ 15-15 & 6-5 \end{array} \right) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right) = I_2\)

\(BA = \left( \begin{array}{cc} (5)(-1)+(2)(3) & (5)(-2)+(2)(5) \\ (-3)(-1)+(-1)(3) & (-3)(-2)+(-1)(5) \end{array} \right) = \left( \begin{array}{cc} -5+6 & -10+10 \\ 3-3 & 6-5 \end{array} \right) = \left( \begin{array}{cc} 1 & 0 \\ 0 & 1 \end{array} \right) = I_2\)

Since \(AB=I_2\) and \(BA=I_2\), \(A\) is invertible and \(B=A^{-1}\).

The matrix inverse is a very powerful concept, allowing us to “undo” matrix multiplication. It’s only defined for square matrices. The identity matrix \(I_n\) plays a crucial role here, similar to how 1 works with reciprocals. Note that both \(AB=I_n\) and \(BA=I_n\) must hold.

Singular Matrices

Some square matrices are not invertible. These are called singular matrices.

Conditions for a Matrix to be Singular:

1. A square matrix with a row (or a column) of zeros is singular.

Note

\(C = \left( \begin{array}{lll}0 & 0 & 0\\ a & b & c\\ d & e & f \end{array} \right)\) is singular because any matrix \(D\) multiplied by \(C\) will result in a matrix with a row of zeros. Thus \(CD \neq I\).

2. A square matrix with a row (or a column) which is a multiple of another row (or column) is singular.

Note

\(E = \left( \begin{array}{ccc}a & b & c\\ 2a & 2b & 2c\\ d & e & f \end{array} \right)\) is singular because \(R_2 = 2R_1\). If \(EF=I\), then \(R_2\) of \(EF\) would be \(2R_1\) of \(EF\), which cannot be the identity matrix.

3. More generally, a square matrix where a row (or column) is a linear combination of other rows (or columns) is singular.

Note

\(G = \left( \begin{array}{cccc}a & b & c\\ d & e & f\\ 2a - 5d & 2b - 5e & 2c - 5f \end{array} \right)\). Here, \(R_3 = 2R_1 - 5R_2\). Thus \(G\) is singular.

These properties provide quick ways to identify if a matrix is singular without performing extensive calculations. A singular matrix implies that certain linear systems associated with it will not have unique solutions. We will later see that a matrix is singular if and only if its determinant is zero.

Uniqueness of the Inverse

If an inverse exists for a matrix, it is unique.

Proposition: If \(B\) and \(\widehat{B}\) are both inverses of an invertible matrix \(A\), then \(B = \widehat{B}\).

Note

Because the inverse is unique, we can confidently use the notation \(A^{-1}\) to refer to the inverse of \(A\).

Proposition: If \(A\) is an invertible matrix, then \(A^{-1}\) is also invertible, and \((A^{-1})^{-1} = A\).

The uniqueness of the inverse is an important property that allows us to treat \(A^{-1}\) as a specific, well-defined entity. The proof is a classic demonstration of using matrix properties like associativity and the identity matrix. The second proposition is a natural consequence, stating that “undoing the undoing” brings you back to the original.

Invertible \(2 \times 2\) Matrices

For \(2 \times 2\) matrices, there’s a simple condition for invertibility and a direct formula for the inverse.

Proposition: A \(2 \times 2\) matrix \(A = \left( \begin{array}{cc} a & b \\ c & d \end{array} \right)\) is invertible if and only if \(ad - bc \neq 0\).

In this case, its inverse \(A^{-1}\) is given by:

\[ A ^ {- 1} = \frac {1}{a d - b c} \left( \begin{array}{c c} d & - b \\ - c & a \end{array} \right). \tag {3.1} \]

Note

The value \(ad - bc\) is called the determinant of \(A\), denoted \(\operatorname{det}(A)\). So, \(A\) is invertible if and only if \(\operatorname{det}(A) \neq 0\).

Invertible \(2 \times 2\) Matrices

Determine if \(A = \left( \begin{array}{ll}5 & 3\\ 7 & 9 \end{array} \right)\) is invertible, and if so, find its inverse.

\(\operatorname{det}(A) = (5)(9) - (3)(7) = 45 - 21 = 24\).

Since \(24 \neq 0\), \(A\) is invertible.

\(A^{-1} = \frac{1}{24} \left( \begin{array}{cc} 9 & -3 \\ -7 & 5 \end{array} \right) = \left( \begin{array}{cc} 9/24 & -3/24 \\ -7/24 & 5/24 \end{array} \right) = \left( \begin{array}{cc} 3/8 & -1/8 \\ -7/24 & 5/24 \end{array} \right)\).

This formula for \(2 \times 2\) matrices is incredibly handy. It’s often the first place students encounter the concept of a determinant, which generalizes to larger square matrices and provides the critical test for invertibility. For larger matrices, the process for finding the inverse is more involved (e.g., Gaussian elimination or adjoint method), but the determinant concept remains central.

Invertibility of Diagonal Matrices

Diagonal matrices have a straightforward condition for invertibility.

Proposition: A diagonal matrix \(A = [a_{ij}]\) is invertible if and only if all its diagonal entries are non-zero (\(a_{ii} \neq 0\) for every \(i\)).

When \(A\) is invertible, its inverse \(A^{-1}\) is a diagonal matrix where each diagonal entry is the reciprocal of the corresponding entry in \(A\).

\[ \text{If } A = \left( \begin{array}{c c c} a & 0 & 0 \\ 0 & b & 0 \\ 0 & 0 & c \end{array} \right) \text{ then } A^{-1} = \left( \begin{array}{c c c} 1/a & 0 & 0 \\ 0 & 1/b & 0 \\ 0 & 0 & 1/c \end{array} \right) \]

Invertibility of Diagonal Matrices

Find the inverse of \(\left( \begin{array}{cccc} 2 & 0 & 0 & 0 \\ 0 & 3 & 0 & 0 \\ 0 & 0 & 5 & 0 \\ 0 & 0 & 0 & -1 \end{array} \right)\).

Since all diagonal entries are non-zero, the matrix is invertible.

Its inverse is \(\left( \begin{array}{cccc} 1/2 & 0 & 0 & 0 \\ 0 & 1/3 & 0 & 0 \\ 0 & 0 & 1/5 & 0 \\ 0 & 0 & 0 & -1 \end{array} \right)\).

This property makes diagonal matrices very easy to work with when it comes to finding inverses. This simplicity extends to triangular matrices as well, where invertibility also depends on non-zero diagonal entries.

Properties of Invertible Matrices

Invertible matrices have several important properties that simplify matrix algebra.

Proposition: Let \(A\) and \(B\) be invertible matrices.

1. Cancellation Laws:
   • \(AB = AC \Rightarrow B = C\) (Left cancellation)
   • \(BA = CA \Rightarrow B = C\) (Right cancellation)
2. Inverse of a Product: The product \(AB\) is invertible, and \((AB)^{-1} = B^{-1} A^{-1}\). (Note the reversed order!)
3. Inverse of a Transpose: \(A^T\) is invertible, and \((A^T)^{-1} = (A^{-1})^T\).
4. Inverse of a Scalar Multiple: For any non-zero scalar \(\alpha\), \(\alpha A\) is invertible, and \((\alpha A)^{-1} = \frac{1}{\alpha} A^{-1}\).

Tip

The cancellation laws are only valid for invertible matrices. If a matrix is singular, you cannot cancel it.

These properties are incredibly powerful. The cancellation laws allow us to simplify equations, similar to scalar algebra, but with the important caveat that the matrix must be invertible. The formula for the inverse of a product is another one that often trips up students if they forget to reverse the order.

Matrix Powers

We can define integer powers for square matrices.

Definition: Let \(A\) be a square matrix.

• Non-negative integer powers:
  • \(A^0 = I\) (Identity matrix)
  • \(A^n = \underbrace{A A \cdots A}_{n \text{ times}}\) for \(n > 0\).
• Negative integer powers (if \(A\) is invertible):
  • \(A^{-n} = (A^{-1})^n = \underbrace{A^{-1} A^{-1} \cdots A^{-1}}_{n \text{ times}}\) for \(n > 0\).

Laws of Exponents (parallel to real numbers): - \(A^r A^s = A^{r+s}\) - \((A^r)^s = A^{rs}\)

Proposition: If \(A\) is an invertible matrix, then \(A^n\) is invertible and \((A^n)^{-1} = (A^{-1})^n\).

Matrix powers are useful in modeling systems that evolve over discrete time steps, such as population growth or Markov chains. The laws of exponents behave as expected, as long as the base matrix is square and invertible for negative powers.

Determinants: Introduction

The determinant is a scalar value that can be computed from the entries of a square matrix. It provides crucial information about the matrix, particularly its invertibility.

• For a \(2 \times 2\) matrix \(A = \left( \begin{array}{cc} a & b \\ c & d \end{array} \right)\), the determinant is \(\operatorname{det}(A) = ad - bc\).
• Key link: \(A\) is invertible if and only if \(\operatorname{det}(A) \neq 0\).

Note

For \(n \times n\) matrices where \(n \geq 3\), the determinant is computed inductively using cofactor expansion.

The determinant is a single number that encapsulates a lot of information about a square matrix. Its relationship with invertibility is one of the most important concepts in linear algebra. While it’s simple for \(2 \times 2\) matrices, it becomes more complex for larger matrices, but the principle remains the same.

Cofactors

To compute determinants for larger matrices, we need minors and cofactors.

Definition: Suppose \(A\) is an \(n \times n\) matrix.

1. The \((i,j)\)-minor of \(A\), denoted \(M_{ij}\), is the determinant of the submatrix formed by deleting the \(i\)th row and \(j\)th column from \(A\).
2. The \((i,j)\)-cofactor of \(A\), denoted \(C_{ij}\), is \((-1)^{i+j}M_{ij}\).

Cofactors

Consider \(A = \left[ \begin{array}{c c c} 1 & 5 & 0 \\ - 3 & 2 & 1 \\ 1 & 2 & 1 \end{array} \right]\).

Find the \((1,1)\)-cofactor \(C_{11}\):

\(M_{11} = \left| \begin{array}{cc} 2 & 1 \\ 2 & 1 \end{array} \right| = (2)(1) - (1)(2) = 0\).

\(C_{11} = (-1)^{1+1}M_{11} = (1)(0) = 0\).

Find the \((2,3)\)-cofactor \(C_{23}\):

\(M_{23} = \left| \begin{array}{cc} 1 & 5 \\ 1 & 2 \end{array} \right| = (1)(2) - (5)(1) = 2 - 5 = -3\).

\(C_{23} = (-1)^{2+3}M_{23} = (-1)(-3) = 3\).

Minors are determinants of smaller matrices derived from the original. Cofactors add a sign based on their position. The \((-1)^{i+j}\) term creates a “checkerboard” pattern of signs, which is a common mnemonic. These are the building blocks for calculating larger determinants.

Determinant via Cofactor Expansion

The determinant of an \(n \times n\) matrix \(A\) (for \(n \geq 3\)) can be found by expanding along any row or column using cofactors.

Expansion along the \(i\)th row: \[ \det (A) = a _ {i 1} C _ {i 1} + a _ {i 2} C _ {i 2} + \dots a _ {i n} C _ {i n} = \sum_ {j = 1} ^ {n} a _ {i j} C i j \]

Expansion along the \(j\)th column: \[ \det (A) = a _ {1 j} C _ {1 j} + a _ {2 j} C _ {2 j} + \dots + a _ {n j} C _ {n j} = \sum_ {k = 1} ^ {n} a _ {k j} C _ {k j}. \]

Determinant via Cofactor Expansion

Find \(\det(A)\) for \(A = \left[ \begin{array}{rrr}1 & 5 & 0\\ -3 & 2 & 1\\ 1 & 2 & 1 \end{array} \right]\) by expanding along the second row.

The checkerboard signs for the second row are -, +, -.

\(C_{21} = (-1)^{2+1} \left| \begin{array}{cc} 5 & 0 \\ 2 & 1 \end{array} \right| = - (5 \cdot 1 - 0 \cdot 2) = -5\)

\(C_{22} = (-1)^{2+2} \left| \begin{array}{cc} 1 & 0 \\ 1 & 1 \end{array} \right| = + (1 \cdot 1 - 0 \cdot 1) = 1\)

\(C_{23} = (-1)^{2+3} \left| \begin{array}{cc} 1 & 5 \\ 1 & 2 \end{array} \right| = - (1 \cdot 2 - 5 \cdot 1) = - (2 - 5) = 3\)

\(\det(A) = a_{21}C_{21} + a_{22}C_{22} + a_{23}C_{23}\)

\(\det(A) = (-3)(-5) + (2)(1) + (1)(3) = 15 + 2 + 3 = 20\).

The key idea here is that you can choose any row or any column for the expansion, and you will always get the same determinant value. A common strategy is to pick the row or column with the most zeros to minimize calculations. The “checkerboard” matrix of signs is a visual aid for \((-1)^{i+j}\).

Determinants of Special Matrices

Certain matrix types have determinants that are easy to compute.

1. Matrices with a zero row/column: If an \(n \times n\) matrix \(A\) has a row of zeros or a column of zeros, then \(\det(A) = 0\).

2. Matrices with linearly dependent rows/columns: If \(A\) has two rows (or columns) such that one is a multiple of the other, then \(\det(A) = 0\). (This is a special case of linear dependence.)

3. Triangular Matrices: If \(A\) is a triangular matrix (upper, lower, or diagonal), then \(\det(A)\) is the product of its diagonal entries. \[ \left| \begin{array}{cccc}4 & 0 & 0 & 5\\ 0 & -2 & 0 & 7\\ 0 & 0 & 3 & 0\\ 0 & 0 & 0 & \frac{1}{2} \end{array} \right| \quad \text{has determinant } (4)(-2)(3)(1/2) = -12. \]

Important

\(\det(I_n) = 1\) because \(I_n\) is a diagonal matrix with all diagonal entries being 1.

These properties are incredibly useful for quickly determining if a matrix is singular (determinant is zero) or for calculating determinants with minimal effort for structured matrices. They reflect deeper theoretical concepts related to linear independence and matrix transformations.

Determinants and Invertibility Revisited

The determinant is the ultimate test for matrix invertibility.

Theorem: For two \(n \times n\) matrices \(A\) and \(B\): \[ \det (A B) = \det (A) \det (B). \]

Corollary: If \(A\) is an \(n \times n\) invertible matrix, then \(\det(A) \neq 0\).

Moreover, \(\det (A ^ {- 1}) = \frac {1}{\det (A)}\).

Proof (Sketch):

We know \(AA^{-1} = I\).

Taking the determinant of both sides: \(\det(AA^{-1}) = \det(I)\).

Using the product rule: \(\det(A)\det(A^{-1}) = 1\).

Since \(\det(I)=1\), this implies \(\det(A) \neq 0\) and \(\det(A^{-1}) = 1/\det(A)\).

Important

A square matrix \(A\) is invertible if and only if \(\det(A) \neq 0\). Equivalently, \(A\) is singular if and only if \(\det(A) = 0\).

This theorem and its corollary solidify the connection between determinants and invertibility. It means that to check if a matrix can be inverted, we just need to calculate its determinant. If it’s zero, no inverse exists. This is a foundational result in linear algebra.

Adjoint of a Matrix (Optional)

The adjoint of a matrix provides a formula for its inverse.

Definition: Let \(A\) be an \(n \times n\) matrix, and \(C_{ij}\) be its \((i,j)\)th cofactor.

The matrix of cofactors from \(A\) is: \[ C = \left[ \begin{array}{c c c c} C _ {1 1} & C _ {1 2} & \dots & C _ {1 n} \\ C _ {2 1} & C _ {2 2} & \dots & C _ {2 n} \\ . & . & \dots & . \\ . & . & \dots & . \\ . & . & \dots & . \\ C _ {n 1} & C _ {n 2} & \dots & C _ {n n} \end{array} \right] \] The adjoint of \(A\), denoted \(\operatorname{adj}(A)\), is the transpose of the matrix of cofactors: \[ \operatorname {adj} (A) = C^T \quad \text{or} \quad (\operatorname {adj} (A)) _ {i j} = C _ {j i}. \]

Adjoint of a Matrix (Optional)

Find \(\operatorname{adj}(A)\) for \(A = \left[ \begin{array}{rrr}1 & 5 & 0\\ -3 & 2 & 1\\ 1 & 2 & 1 \end{array} \right]\). (We found \(\det(A)=20\) earlier).

Cofactor matrix \(C = \left[ \begin{array}{r r r} 0 & 4 & - 8 \\ - 5 & 1 & 3 \\ 5 & - 1 & 1 7 \end{array} \right]\).

\(\operatorname{adj}(A) = C^T = \left[ \begin{array}{r r r} 0 & - 5 & 5 \\ 4 & 1 & - 1 \\ - 8 & 3 & 1 7 \end{array} \right]\).

The adjoint is a crucial concept because it directly links the inverse of a matrix to its determinant and cofactors. While calculating the adjoint for large matrices can be tedious, it’s a theoretically important construction.

Inverse Formula via Adjoint (Optional)

The adjoint matrix gives a direct formula for the inverse of an invertible matrix.

Theorem: Let \(A\) be an \(n \times n\) square matrix. Then: \[ A \operatorname{adj} (A) = \det (A) I. \]

Theorem: If \(A\) is an \(n \times n\) invertible matrix, then: \[ A ^ {- 1} = \frac {1}{\det (A)} \operatorname {adj} (A). \]

Inverse Formula via Adjoint (Optional)

Find the inverse of \(A = \left[ \begin{array}{rrr}1 & 5 & 0\\ -3 & 2 & 1\\ 1 & 2 & 1 \end{array} \right]\) using the adjoint.

We found \(\det(A) = 20\) and \(\operatorname{adj}(A) = \left[ \begin{array}{r r r} 0 & - 5 & 5 \\ 4 & 1 & - 1 \\ - 8 & 3 & 1 7 \end{array} \right]\).

\[ A^{-1} = \frac{1}{20} \left[ \begin{array}{r r r} 0 & - 5 & 5 \\ 4 & 1 & - 1 \\ - 8 & 3 & 1 7 \end{array} \right] = \left[ \begin{array}{r r r} 0 & -1/4 & 1/4 \\ 1/5 & 1/20 & -1/20 \\ -2/5 & 3/20 & 17/20 \end{array} \right]. \]

This formula is theoretically elegant, offering a direct way to compute the inverse using determinants and cofactors. For practical numerical computation with large matrices, Gaussian elimination is often more efficient, but the adjoint formula provides valuable insight into the structure of the inverse.

Cramer’s Rule

Cramer’s Rule offers a method to solve systems of linear equations using determinants, when the coefficient matrix is invertible.

Consider a system \(\mathbf{Ax} = \mathbf{b}\) of \(n\) linear equations in \(n\) unknowns.

If \(\det(A) \neq 0\), then the system has a unique solution given by: \[ x _ {j} = \frac {\det (A _ {j})}{\det (A)}, \quad j = 1, 2, \ldots , n \] where \(A_j\) is the matrix formed by replacing the \(j\)th column of \(A\) with the vector \(\mathbf{b}\).

\[ A _ {j} = \left[ \begin{array}{c c c c c c c} a _ {1 1} & \dots & a _ {1 j - 1} & \mathbf{b _ {1}} & a _ {1 j + 1} & \dots & a _ {1 n} \\ a _ {2 1} & \dots & a _ {2 j - 1} & \mathbf{b _ {2}} & a _ {2 j + 1} & \dots & a _ {2 n} \\ \vdots & & \vdots & \vdots & \vdots & & \vdots \\ a _ {n 1} & \dots & a _ {n j - 1} & \mathbf{b _ {n}} & a _ {n j + 1} & \dots & a _ {n n} \end{array} \right] \]

Important

Cramer’s Rule is only applicable if the coefficient matrix \(A\) is square and \(\det(A) \neq 0\).

Cramer’s Rule is theoretically significant because it provides an explicit formula for the solution components. While computationally intensive for large systems, it’s very useful for small systems (like \(2 \times 2\) or \(3 \times 3\)) and for deriving properties of solutions without actually solving the system.

Cramer’s Rule Example

Let’s apply Cramer’s Rule to a \(2 \times 2\) system.

Solve the linear system: \[ \begin{array}{l} 7 x _ {1} - 2 x _ {2} = 3 \\ 3 x _ {1} + x _ {2} = 5 \\ \end{array} \] This can be written as \(A\mathbf{x} = \mathbf{b}\) where \(A = \left[ \begin{array}{cc} 7 & -2 \\ 3 & 1 \end{array} \right]\) and \(\mathbf{b} = \left[ \begin{array}{c} 3 \\ 5 \end{array} \right]\).

1. Calculate \(\det(A)\): \(\det(A) = (7)(1) - (-2)(3) = 7 - (-6) = 13\). Since \(\det(A) = 13 \neq 0\), Cramer’s Rule is applicable.

2. Form \(A_1\) and calculate \(\det(A_1)\): Replace column 1 of \(A\) with \(\mathbf{b}\). \(A_1 = \left[ \begin{array}{cc} 3 & -2 \\ 5 & 1 \end{array} \right]\). \(\det(A_1) = (3)(1) - (-2)(5) = 3 - (-10) = 13\).

3. Form \(A_2\) and calculate \(\det(A_2)\): Replace column 2 of \(A\) with \(\mathbf{b}\). \(A_2 = \left[ \begin{array}{cc} 7 & 3 \\ 3 & 5 \end{array} \right]\). \(\det(A_2) = (7)(5) - (3)(3) = 35 - 9 = 26\).

4. Find \(x_1\) and \(x_2\): \(x_1 = \frac{\det(A_1)}{\det(A)} = \frac{13}{13} = 1\). \(x_2 = \frac{\det(A_2)}{\det(A)} = \frac{26}{13} = 2\).

The unique solution is \(x_1=1, x_2=2\).

This example clearly illustrates the step-by-step application of Cramer’s Rule. It’s a systematic process, and as long as the determinant of the coefficient matrix is non-zero, it will yield the unique solution. For systems larger than \(3 \times 3\), calculating all these determinants can become very cumbersome.

Key Takeaways

1. Matrices are fundamental: Rectangular arrays representing data and transformations.
2. Notation & Terminology: Understand matrix size (\(m \times n\)), entries (\(a_{ij}\)), and special types (square, identity, diagonal, triangular).
3. Basic Operations:
   • Addition/Subtraction: Entry-wise, requires same dimensions.
   • Scalar Multiplication: Multiply each entry by the scalar.
4. Matrix Multiplication: Row-column dot product. Order matters (\(AB \neq BA\)). Requires inner dimensions to match.
5. Transpose: Interchanging rows and columns (\(A^T\)). Property \((AB)^T = B^T A^T\).
6. Matrix Inverse (\(A^{-1}\)): Exists only for square matrices where \(AA^{-1} = A^{-1}A = I\). Unique if it exists.
7. Determinants (\(\det(A)\)): Scalar value, calculated via cofactor expansion for \(n \geq 3\).
8. Invertibility Test: A square matrix \(A\) is invertible if and only if \(\det(A) \neq 0\).
9. Cramer’s Rule: Solves linear systems \(A\mathbf{x} = \mathbf{b}\) if \(A\) is square and \(\det(A) \neq 0\), using ratios of determinants.

These are the most important concepts to grasp from this chapter. They form the foundation for more advanced topics in linear algebra and its applications. Ensure you are comfortable with these definitions and properties.

Key Equations

Equation	Description
\((A + B)_{ij} = A_{ij} + B_{ij}\)	Matrix addition (entry-wise)
\((\alpha A)_{ij} = \alpha (A_{ij})\)	Scalar multiplication (entry-wise)
\((AB)_{ij} = \sum_{k=1}^{r} A_{ik} B_{kj}\)	Matrix multiplication (row-column dot product)
\((A^T)_{ij} = A_{ji}\)	Transpose of a matrix
\(AA^{-1} = I\) and \(A^{-1}A = I\)	Definition of matrix inverse
\(A^{-1} = \frac{1}{ad-bc} \left( \begin{array}{cc} d & -b \\ -c & a \end{array} \right)\)	Inverse of a \(2 \times 2\) matrix \(A=\left(\begin{smallmatrix} a & b \\ c & d \end{smallmatrix}\right)\)
\(\det(A) = ad-bc\)	Determinant of a \(2 \times 2\) matrix \(A=\left(\begin{smallmatrix} a & b \\ c & d \end{smallmatrix}\right)\)
\(\det(A) = \sum_{j=1}^{n} a_{ij}C_{ij}\)	Cofactor expansion along row \(i\)
\(\det(AB) = \det(A)\det(B)\)	Determinant of a matrix product
\(A^{-1} = \frac{1}{\det(A)} \operatorname{adj}(A)\)	Inverse using the adjoint matrix (if \(\det(A) \neq 0\))
\(x_j = \frac{\det(A_j)}{\det(A)}\)	Cramer’s Rule for solving \(A\mathbf{x} = \mathbf{b}\)

This table summarizes the core mathematical formulas discussed. Make sure you understand what each equation represents and when to apply it.

Key Terms

Term	Definition
Matrix	A rectangular array of numbers or mathematical objects.
\((i,j)\)th Entry	The element located in the \(i\)th row and \(j\)th column of a matrix.
Size (\(m \times n\))	Describes a matrix with \(m\) rows and \(n\) columns.
Row Matrix (Vector)	A matrix with only one row.
Column Matrix (Vector)	A matrix with only one column.
Zero Matrix	A matrix where all entries are zero.
Square Matrix	A matrix where the number of rows equals the number of columns (\(m=n\)).
Diagonal Entries	Entries \(a_{ii}\) along the main diagonal of a square matrix.
Identity Matrix (\(I_n\))	An \(n \times n\) square matrix with 1s on the main diagonal and 0s elsewhere.
Diagonal Matrix	A square matrix where all off-diagonal entries are zero.
Triangular Matrix	A square matrix with all entries either above (upper triangular) or below (lower triangular) the main diagonal being zero.
Scalar	A real or complex number used in matrix operations.
Transpose (\(A^T\))	The matrix obtained by interchanging the rows and columns of \(A\).
Invertible (Non-singular)	A square matrix \(A\) for which an inverse matrix \(A^{-1}\) exists.
Not Invertible (Singular)	A square matrix for which no inverse matrix exists.
Determinant (\(\det(A)\))	A scalar value calculated from the entries of a square matrix, indicating its invertibility.
Minor (\(M_{ij}\))	The determinant of the submatrix remaining after deleting the \(i\)th row and \(j\)th column.
Cofactor (\(C_{ij}\))	The minor \(M_{ij}\) multiplied by \((-1)^{i+j}\).
Adjoint (\(\operatorname{adj}(A)\))	The transpose of the matrix of cofactors.
Cramer’s Rule	A formula for solving a system of linear equations using determinants.

Familiarize yourself with these terms. A strong vocabulary is essential for understanding and communicating effectively in linear algebra.