Linear Algebra

2.3 Properties of Determinants; Cramer’s Rule

Imron Rosyadi

Linear Algebra

2.3 Properties of Determinants; Cramer’s Rule

Imron Rosyadi

Introduction & Objectives

Today we’ll explore fundamental properties of determinants that are essential for deeper matrix analysis and, crucially, for practical tools like finding matrix inverses and solving linear systems.

Learning Objectives

• Understand how scalars, matrix sums, and products affect determinants.
• Define the adjoint of a matrix and use it to find \(A^{-1}\).
• Learn Cramer’s Rule for solving \(A\mathbf{x} = \mathbf{b}\).
• Consolidate the major theorems linking invertibility, determinants, and system solutions.

ECE Relevance

These topics are fundamental for advanced signal processing, control system design, robotics, and circuit analysis, where efficient computation of inverses and system solutions is paramount.

In ECE, we frequently encounter large systems of linear equations or matrices representing system dynamics. Efficiently solving these or determining matrix properties is not just academic; it directly impacts the performance, stability, and feasibility of engineered systems. For example, knowing if a determinant is zero for a control system matrix quickly tells us if the system is uncontrollable or unobservable.

Basic Properties: Scalar Multiplication and Sums

Let \(A\) and \(B\) be \(n \times n\) matrices and \(k\) be any scalar.

Scalar Multiplication of a Matrix

Scaling an entire matrix \(A\) by \(k\) affects its determinant by \(k^n\). \[ \operatorname *{det}(k A) = k^{n}\operatorname *{det}(A) \] Reason: Each of the \(n\) rows of \(kA\) has a common factor of \(k\). Since a common factor from any individual row can be factored out of the determinant, factoring \(k\) from each of the \(n\) rows results in \(n\) factors of \(k\).

Example for \(n=3\): \[ \left| \begin{array}{lll}k a_{11} & k a_{12} & k a_{13} \\ k a_{21} & k a_{22} & k a_{23} \\ k a_{31} & k a_{32} & k a_{33} \end{array} \right| = k^{3}\left| \begin{array}{lll}a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{array} \right| \]

Basic Properties: Scalar Multiplication and Sums

Determinant of a Sum of Matrices

Unlike many matrix operations, there is no simple relationship between \(\operatorname{det}(A+B)\), \(\operatorname{det}(A)\), and \(\operatorname{det}(B)\). In general, \(\operatorname{det}(A+B) \neq \operatorname{det}(A) + \operatorname{det}(B)\).

Example 1: \(\operatorname *{det}(A + B) \neq \operatorname *{det}(A) + \operatorname *{det}(B)\)

\[ A = \left[ \begin{array}{ll}1 & 2 \\ 2 & 5 \end{array} \right], \quad B = \left[ \begin{array}{ll}3 & 1 \\ 1 & 3 \end{array} \right] \] \(\operatorname *{det}(A) = (1)(5) - (2)(2) = 1\) \(\operatorname *{det}(B) = (3)(3) - (1)(1) = 8\) \(A + B = \left[ \begin{array}{ll}4 & 3 \\ 3 & 8 \end{array} \right]\) \(\operatorname *{det}(A+B) = (4)(8) - (3)(3) = 32 - 9 = 23\) Here, \(1 + 8 \neq 23\).

The scalar multiplication property for determinants is highly useful. For instance, in scaling system parameters (e.g., gains in a control system or resistor values in a circuit), this helps understand how the overall system scaling impacts the determinant, which is linked to system stability and invertibility. The non-additivity of determinants is a common point of confusion; always emphasize that determinants are not linear functions like that.

Sums of Determinants with One Different Row/Column

While \(\operatorname{det}(A+B) \neq \operatorname{det}(A) + \operatorname{det}(B)\) is true in general, there’s a specific case where a useful addition property holds.

Theorem 2.3.1: Determinant Sums (One Row/Column Different)

Let \(A, B, C\) be \(n \times n\) matrices that differ only in a single row (say the \(r\)-th row). If the \(r\)-th row of \(C\) is the sum of the \(r\)-th rows of \(A\) and \(B\), then: \[ \operatorname *{det}(C) = \operatorname *{det}(A) + \operatorname *{det}(B) \] The same result holds for columns.

Example: \(2 \times 2\) Illustration

If \(A = \left[ \begin{array}{ll}a_{11} & a_{12} \\ a_{21} & a_{22} \end{array} \right]\) and \(B = \left[ \begin{array}{ll}a_{11} & a_{12} \\ b_{21} & b_{22} \end{array} \right]\), then \(C = \left[ \begin{array}{cc}{a_{11}} & {a_{12}}\\ {a_{21} + b_{21}} & {a_{22} + b_{22}} \end{array} \right]\).

Calculating:

\(\operatorname{det}(A) + \operatorname{det}(B) = (a_{11}a_{22} - a_{12}a_{21}) + (a_{11}b_{22} - a_{12}b_{21})\)

\(= a_{11}(a_{22} + b_{22}) - a_{12}(a_{21} + b_{21})\)

This is exactly \(\operatorname{det}(C)\), expanded along the second row.

This property is less intuitive but can sometimes simplify calculations if you can decompose a matrix into two simpler ones that differ by only one row or column. It’s a precise condition, not a general one. While less frequently applied directly than other properties in typical ECE problem-solving, it is fundamental in proving other more advanced determinant properties and in some theoretical developments in linear algebra.

Determinant of a Matrix Product

One of the most elegant and powerful properties of determinants is how they behave with matrix multiplication.

Theorem 2.3.4: Determinant of a Product

If \(A\) and \(B\) are square matrices of the same size, then: \[ \operatorname *{det}(A B) = \operatorname *{det}(A)\operatorname *{det}(B) \]

Proof sketch:

1. Lemma 2.3.2 (Elementary Matrices): First, show for an elementary matrix \(E\) and any matrix \(B\), \(\operatorname{det}(EB) = \operatorname{det}(E)\operatorname{det}(B)\).

   • This is shown by considering the three types of EROs and how they affect the determinant (Theorem 2.2.3 and 2.2.4).

Determinant of a Matrix Product

2. Determinant Test for Invertibility (Theorem 2.3.3): A square matrix \(A\) is invertible if and only if \(\operatorname{det}(A) \ne 0\).

   • This critical theorem implies that if \(\operatorname{det}(A)=0\), then \(AB\) is also not invertible, meaning \(\operatorname{det}(AB)=0\), so \(0 = 0 \cdot \operatorname{det}(B)\).

3. General Case: If \(A\) is invertible, it can be written as a product of elementary matrices \(A = E_1 E_2 \cdots E_r\). Then, using Lemma 2.3.2 repeatedly:

   \(\operatorname{det}(AB) = \operatorname{det}(E_1 E_2 \cdots E_r B) = \operatorname{det}(E_1)\operatorname{det}(E_2)\cdots\operatorname{det}(E_r)\operatorname{det}(B)\)

   And \(\operatorname{det}(A) = \operatorname{det}(E_1 E_2 \cdots E_r) = \operatorname{det}(E_1)\operatorname{det}(E_2)\cdots\operatorname{det}(E_r)\).

   Combining these gives \(\operatorname{det}(AB) = \operatorname{det}(A)\operatorname{det}(B)\).

This theorem is incredibly useful in ECE, especially in transform domains (e.g., Fourier, Laplace) where operations might involve matrix products. For example, if you’re chaining linear transformations (like processing a signal through multiple filters represented by matrices), the overall scaling effect on the “volume” (determinant) is simply the product of the individual scalings. This property is also critical when dealing with system stability where product matrices arise, e.g., in cascaded control systems.

Interactive Example: Determinant of a Product

Let’s verify \(\operatorname *{det}(AB) = \operatorname *{det}(A)\operatorname *{det}(B)\) with an example.

Given matrices: \[ A={\left[\begin{array}{l l}{3}&{1}\\ {2}&{1}\end{array}\right]},\quad B={\left[\begin{array}{l l}{-1}&{3}\\ {5}&{8}\end{array}\right]} \]

1. Calculate \(\operatorname *{det}(A)\) and \(\operatorname *{det}(B)\).
2. Calculate \(AB\).
3. Calculate \(\operatorname *{det}(AB)\).
4. Compare results.

The example clearly shows that \(\operatorname{det}(A)=1\), \(\operatorname{det}(B)=-23\), and \(\operatorname{det}(AB)=-23\), confirming the theorem. Play around with different matrices in the interactive code. This hands-on verification strengthens the understanding of this very important determinant property.

Determinant of an Inverse

Another useful property relates the determinant of an invertible matrix to the determinant of its inverse.

Theorem 2.3.5: Determinant of Inverse

If \(A\) is invertible, then: \[ \operatorname *{det}(A^{-1}) = \frac{1}{\operatorname*{det}(A)} \]

Proof sketch:

Since \(A^{-1}A = I\), by Theorem 2.3.4, we have \(\operatorname{det}(A^{-1}A) = \operatorname{det}(A^{-1})\operatorname{det}(A)\).

Also, \(\operatorname{det}(I) = 1\).

Thus, \(\operatorname{det}(A^{-1})\operatorname{det}(A) = 1\).

Since \(A\) is invertible, \(\operatorname{det}(A) \ne 0\), so we can divide by \(\operatorname{det}(A)\) to get \(\operatorname{det}(A^{-1}) = 1/\operatorname{det}(A)\).

This theorem reinforces the link between invertibility (\(\operatorname{det}(A) \ne 0\)) and the determinant. Its direct application is when you have to calculate a determinant of an inverse, you don’t actually have to calculate the inverse matrix itself. For system stability analysis or filter design in ECE, understanding the determinant of \(A^{-1}\) can indicate how “robust” a system is to perturbations.

Adjoint of a Matrix

The adjoint of a matrix provides a direct formula for the inverse, extending the \(2 \times 2\) formula.

DEFINITION 1: Adjoint of \(A\), \(\operatorname{adj}(A)\) If \(A\) is any \(n \times n\) matrix and \(C_{ij}\) is the cofactor of \(a_{ij}\), then the matrix of cofactors is: \[ \left[ \begin{array}{c c c c}{C_{11}} & {C_{12}} & \dots & {C_{1n}}\\ {C_{21}} & {C_{22}} & \dots & {C_{2n}}\\ \vdots & \vdots & & \vdots \\ {C_{n1}} & {C_{n2}} & \dots & {C_{n n}} \end{array} \right] \] The adjoint of \(A\), denoted \(\operatorname{adj}(A)\), is the transpose of this matrix of cofactors. \[ \operatorname{adj}(A) = [\text{Matrix of Cofactors}]^T \]

Adjoint of a Matrix

Important Property: Off-Diagonal Cofactor Sums

If you multiply the entries in any row (or column) of \(A\) by the corresponding cofactors from a different row (or column), the sum is always zero. Example (from text): \(3C_{21} + 2C_{22} + (-1)C_{23} = 0 \quad\) (entries from row 1, cofactors from row 2).

Theorem 2.3.6: Formula for \(A^{-1}\)

If \(A\) is an invertible matrix, then: \[ A^{-1} = \frac{1}{\operatorname*{det}(A)} \operatorname {adj}(A) \]

The adjoint formula is a foundational result for understanding matrix inverses in a theoretical sense. While numerically, Gaussian elimination is much faster for high-dimensional matrices, the adjoint formula is invaluable for smaller systems or symbolic computations, for example, in deriving transfer functions in control systems or impedance matrices in circuits. The property that off-diagonal cofactor sums are zero is key to the proof and a beautiful aspect of determinant theory.

Example: Finding Inverse using Adjoint

Let’s find the inverse of \(A\) using the adjoint formula. \[ A = \left[ \begin{array}{rrr}3 & 2 & -1 \\ 1 & 6 & 3 \\ 2 & -4 & 0 \end{array} \right] \]

From previous calculations (Example 5. in original text): \(\operatorname *{det}(A) = 64\).

Cofactors of \(A\): \(C_{11}=12, \quad C_{12}=6, \quad C_{13}=-16\) \(C_{21}=4, \quad C_{22}=2, \quad C_{23}=16\) \(C_{31}=12, \quad C_{32}=-10, \quad C_{33}=16\)

Matrix of Cofactors: \[ \left[ \begin{array}{rrr}12 & 6 & -16 \\ 4 & 2 & 16 \\ 12 & -10 & 16 \end{array} \right] \]

Example: Finding Inverse using Adjoint

Adjoint of \(A\) (transpose of matrix of cofactors): \[ \operatorname {adj}(A) = \left[ \begin{array}{rrr}12 & 4 & 12 \\ 6 & 2 & -10 \\ -16 & 16 & 16 \end{array} \right] \]

Inverse of \(A\): \[ A^{-1} = \frac{1}{\operatorname*{det}(A)}\mathrm{adj}(A) = \frac{1}{64}\left[ \begin{array}{rrr}{12} & 4 & {12}\\ 6 & {2} & {-10}\\ {-16} & {16} & {16} \end{array} \right] = \left[ \begin{array}{rrr}\frac{12}{64} & \frac{4}{64} & \frac{12}{64}\\ \frac{6}{64} & \frac{2}{64} & -\frac{10}{64}\\ -\frac{16}{64} & \frac{16}{64} & \frac{16}{64} \end{array} \right] = \left[ \begin{array}{rrr}\frac{3}{16} & \frac{1}{16} & \frac{3}{16}\\ \frac{3}{32} & \frac{1}{32} & -\frac{5}{32}\\ -\frac{1}{4} & \frac{1}{4} & \frac{1}{4} \end{array} \right] \]

Walk through the calculation of cofactors and then forming the adjoint carefully, emphasizing the transpose step. Then show the final division by the determinant. This method provides a clear, step-by-step approach for computing the inverse of \(3 \times 3\) matrices, which is essential practice. In ECE, for small systems (like a 3-node circuit or a 3-state control system), this formula is viable.

Interactive Verification: Adjoint Inverse

Verify the inverse calculated using the adjoint method with NumPy.

Cramer’s Rule for Solving Linear Systems

Cramer’s Rule offers a general formula for the solution of \(A\mathbf{x} = \mathbf{b}\) when \(A\) is invertible. While powerful conceptually, it’s often slower than Gaussian elimination for larger systems.

Theorem 2.3.7: Cramer’s Rule

If \(A\mathbf{x} = \mathbf{b}\) is a system of \(n\) linear equations in \(n\) unknowns such that \(\operatorname *{det}(A) \neq 0\), then the system has a unique solution given by: \[ x_{1} = \frac{\operatorname*{det}(A_{1})}{\operatorname*{det}(A)}, x_{2} = \frac{\operatorname*{det}(A_{2})}{\operatorname*{det}(A)}, \ldots , x_{n} = \frac{\operatorname*{det}(A_{n})}{\operatorname*{det}(A)} \] where \(A_{j}\) is the matrix obtained by replacing the \(j\)-th column of \(A\) with the column vector \(\mathbf{b}\).

Proof sketch:

The proof uses the formula \(\mathbf{x} = A^{-1}\mathbf{b} = \frac{1}{\operatorname{det}(A)}\operatorname{adj}(A)\mathbf{b}\).

Expanding this matrix multiplication term by term reveals that the \(j\)-th component \(x_j\) is precisely \(\frac{b_1 C_{1j} + b_2 C_{2j} + \dots + b_n C_{nj}}{\operatorname{det}(A)}\).

The numerator is precisely the cofactor expansion of \(\operatorname{det}(A_j)\) along its \(j\)-th column.

Cramer’s Rule is a classic result with conceptual importance. In ECE, it’s particularly useful for small systems (like 2x2 or 3x3) where you might need an analytical solution or when parameters are involved, allowing calculation of a symbolic rather than just numerical result. In signal processing, for example, if you have a transfer function denominator polynomial, calculating its roots often involves solving systems of equations, and Cramer’s rule can sometimes provide the explicit functions for these roots. For large systems, however, direct numerical solvers are preferable due to computational cost.

Example: Using Cramer’s Rule

Solve the following system using Cramer’s Rule: \[ \begin{array}{r l} & {x_{1} + 0x_{2} + 2x_{3} = 6}\\ & {-3x_{1} + 4x_{2} + 6x_{3} = 30}\\ & {-x_{1} - 2x_{2} + 3x_{3} = 8}\end{array} \]

The coefficient matrix \(A\) and the augmented matrices \(A_1, A_2, A_3\) are: \[ A=\left[\begin{array}{crc}{1}&{0}&{2}\\ {-3}&{4}&{6}\\ {-1}&{-2}&{3}\end{array}\right], \quad A_{1}=\left[\begin{array}{crc}{6}&{0}&{2}\\ {30}&{4}&{6}\\ {8}&{-2}&{3}\end{array}\right] \] \[ A_{2}=\left[\begin{array}{crc}{1}&{6}&{2}\\ {-3}&{30}&{6}\\ {-1}&{8}&{3}\end{array}\right], \quad A_{3}=\left[\begin{array}{crc}{1}&{0}&{6}\\ {-3}&{4}&{30}\\ {-1}&{-2}&{8}\end{array}\right] \]

Example: Using Cramer’s Rule

First, find \(\operatorname{det}(A)\): \(\operatorname{det}(A) = 1(12 - (-12)) - 0(\dots) + 2(6 - (-4)) = 1(24) + 2(10) = 24 + 20 = 44\).

Next, calculate determinants of \(A_1, A_2, A_3\):
\(\operatorname{det}(A_1) = 6(12 - (-12)) - 0(\dots) + 2(-60 - 32) = 6(24) + 2(-92) = 144 - 184 = -40\).
\(\operatorname{det}(A_2) = 1(90 - 48) - 6(-9 - (-6)) + 2(-24 - (-30)) = 1(42) - 6(-3) + 2(6) = 42 + 18 + 12 = 72\).
\(\operatorname{det}(A_3) = 1(32 - (-60)) - 0(\dots) + 6(6 - (-4)) = 1(92) + 6(10) = 92 + 60 = 152\).

Finally, compute \(x_1, x_2, x_3\):
\(x_{1} = \frac{\operatorname*{det}(A_{1})}{\operatorname*{det}(A)} = \frac{-40}{44} = \frac{-10}{11}\)
\(x_{2} = \frac{\operatorname*{det}(A_{2})}{\operatorname*{det}(A)} = \frac{72}{44} = \frac{18}{11}\)
\(x_{3} = \frac{\operatorname*{det}(A_{3})}{\operatorname*{det}(A)} = \frac{152}{44} = \frac{38}{11}\)

This manual calculation is quite tedious but demonstrates the mechanics of Cramer’s Rule. Emphasize that each step involves calculating a determinant. While it’s great for understanding, for realistic engineering systems, numerical methods like Gauss-Jordan or LU decomposition are orders of magnitude faster.

Interactive Verification: Cramer’s Rule

Let’s use Pyodide to verify our solution from the previous example.

Equivalent Statements: The Big Picture

This theorem unifies many core concepts in Linear Algebra that relate to matrix invertibility, system solutions, and rank.

Theorem 2.3.8: Equivalent Statements

If \(A\) is an \(n \times n\) matrix, then the following statements are equivalent:

1. \(A\) is invertible.
2. \(A\mathbf{x} = \mathbf{0}\) has only the trivial solution (\(\mathbf{x} = \mathbf{0}\)).
3. The reduced row echelon form of \(A\) is \(I_{n}\).
4. \(A\) can be expressed as a product of elementary matrices.
5. \(A\mathbf{x} = \mathbf{b}\) is consistent for every \(n \times 1\) matrix \(\mathbf{b}\).
6. \(A\mathbf{x} = \mathbf{b}\) has exactly one solution for every \(n \times 1\) matrix \(\mathbf{b}\).
7. \(\operatorname *{det}(A)\neq 0\).

Equivalent Statements: The Big Picture

ECE Significance

This theorem is a cornerstone for understanding and diagnosing linear systems in engineering.

• System Solvability: In circuit analysis or control, if \(\det(A) = 0\), the system might have no unique solution, indicating a design flaw or dependency.

• Control & State Space: An invertible system matrix ensures controllability or observability (depends on specific matrix, e.g., controllability matrix), crucial for designing stable and responsive systems.

• Hardware/Software Implications: Non-invertible matrices often lead to numerical instability or errors in computational simulations and algorithms used in embedded systems or scientific computing.

• Stability Analysis In control systems, if a system matrix \(A\) needs to be inverted (e.g., to find state transitions), a non-zero determinant is essential. The value \(1/\operatorname{det}(A)\) provides insight into how sensitive the inverse operation might be to small changes in \(A\). A very small \(\operatorname{det}(A)\) implies a very large \(\operatorname{det}(A^{-1})\), indicating the system might be close to singularity or numerically ill-conditioned, which is a critical concern in signal processing and numerical simulations.

Emphasize this theorem as a grand unifying theory up to this point in Linear Algebra. For ECE students, it’s not just a list to memorize; it’s a set of diagnostic tools. For example, if you’re trying to find an inverse and \(\operatorname{det}(A)=0\), you immediately know A is not invertible, which in turn tells you Ax=0 has non-trivial solutions, and its RREF is not the identity.

Summary and Key Takeaways

Properties of Determinants

• Scalar Multiplication: \(\operatorname{det}(kA) = k^n \operatorname{det}(A)\).
• Sums (Specific Case): \(\operatorname{det}(C) = \operatorname{det}(A) + \operatorname{det}(B)\) if only one row/column differs.
• Products: \(\operatorname{det}(AB) = \operatorname{det}(A)\operatorname{det}(B)\).
• Inverse: \(\operatorname{det}(A^{-1}) = 1/\operatorname{det}(A)\).

Advanced Tools

• Adjoint Formula for Inverse: \(A^{-1} = \frac{1}{\operatorname*{det}(A)} \operatorname {adj}(A)\).
• Cramer’s Rule for Linear Systems: \(x_j = \frac{\operatorname*{det}(A_j)}{\operatorname*{det}(A)}\).

Summary and Key Takeaways

Unifying Theorem

• Equivalent Statements (Theorem 2.3.8): Invertibility, unique solutions, RREF to \(I_n\), product of elementary matrices, and non-zero determinant are all interconnected.

Importance for ECE

These concepts are vital for:

• Analyzing stability and behavior of linear systems.
• Efficiently solving systems of equations in hardware and software.
• Understanding the theoretical underpinnings of advanced numerical algorithms.

Today’s lecture has armed you with crucial insights into the behavior of determinants under various matrix operations. These aren’t just abstract mathematical facts; they are foundational principles that will guide your understanding and problem-solving in numerous ECE applications, from signal processing to control theory and beyond. Master these properties, and you’ll gain a deeper appreciation for the logic and power of linear algebra.