Linear Algebra

5.2 Diagonalization of Matrices

Imron Rosyadi

Diagonalization of Matrices: Simplifying Systems in ECE

Why Diagonalization?

Diagonalization is a powerful technique in Linear Algebra that simplifies complex matrix operations.

It involves transforming a matrix into a special diagonal form.

Benefits:

• Simplifies computations: Especially for matrix powers.
• Reveals system behavior: Eigenvalues and eigenvectors provide insights into dynamics.
• Decouples systems: Transforms coupled systems into independent ones, easing analysis.

Core Idea:

Imagine viewing a complex system from a “better” perspective. Diagonalization finds this perspective (a new basis) where the system’s behavior becomes much simpler.

This “simpler” form is a diagonal matrix, where non-diagonal elements are zero.

Note

In ECE, diagonalization is crucial for analyzing:

• Control Systems
• Signal Processing
• Circuit Analysis
• Vibration Analysis

Today, we’ll explore why diagonalization is so important, how to achieve it, and its direct applications in electrical and computer engineering. Think of it as finding the natural modes or fundamental components of a system.

Similarity Transformations

A square matrix \(B\) is similar to \(A\) if there exists an invertible matrix \(P\) such that:

\[ B = P^{-1}AP \]

This transformation, \(A \to P^{-1}AP\), is called a similarity transformation.

Intuition:

A similarity transformation is like changing the coordinate system in which a linear transformation (represented by matrix \(A\)) is viewed. The matrix \(P\) acts as a “change of basis” matrix.

• \(P\): Transforms vectors from the new basis to the standard basis.
• \(P^{-1}\): Transforms vectors from the standard basis to the new basis.

Tip

Similarity transformations preserve fundamental properties of the matrix, even though the matrix elements might change.

Similarity Transformations

Diagram: Change of Basis

flowchart LR
    subgraph Standard Basis
        X_std["Vector X"] --> A_std["A (Standard Basis)"]
    end

    subgraph New Basis
        X_new["Vector X'"] --> A_new["D (New Basis)"]
    end

    X_std -- "P^{-1}" --> X_new
    X_new -- "P" --> X_std

    A_std -- "P^{-1} (Change of Basis)" --> D["D = P^{-1}AP"]
    D -- "P (Change of Basis)" --> A_std

Think of it like describing the same object in different languages. The object itself doesn’t change, just its description. In linear algebra, similarity transformations describe the same linear operator using different bases. The matrix \(P\) is key here; it’s the bridge between the two representations.

Similarity Invariants

Properties preserved under similarity transformations are called similarity invariants.

These are crucial because they tell us what fundamental aspects of a linear transformation remain the same, regardless of the chosen basis.

Key Invariants:

• Determinant: \(\operatorname{det}(B) = \operatorname{det}(A)\)
• Trace: \(\operatorname{tr}(B) = \operatorname{tr}(A)\)
• Rank: \(\operatorname{rank}(B) = \operatorname{rank}(A)\)
• Nullity: \(\operatorname{nullity}(B) = \operatorname{nullity}(A)\)
• Characteristic Polynomial: \(p_B(\lambda) = p_A(\lambda)\)
• Eigenvalues: \(A\) and \(P^{-1}AP\) have the same eigenvalues.

Important

If two matrices are similar, they share these fundamental characteristics.

Similarity Invariants

Interactive Example:

Enter a matrix \(A\) and an invertible matrix \(P\). Observe how key invariants remain the same for \(A\) and \(P^{-1}AP\).

viewof a11 = Inputs.number({value: 2, label: "A[0,0]"});
viewof a12 = Inputs.number({value: 1, label: "A[0,1]"});
viewof a21 = Inputs.number({value: 1, label: "A[1,0]"});
viewof a22 = Inputs.number({value: 2, label: "A[1,1]"});

viewof p11 = Inputs.number({value: 1, label: "P[0,0]"});
viewof p12 = Inputs.number({value: 1, label: "P[0,1]"});
viewof p21 = Inputs.number({value: -1, label: "P[1,0]"});
viewof p22 = Inputs.number({value: 1, label: "P[1,1]"});

The interactive example demonstrates this principle. You can change the entries of matrix A and P. As long as P is invertible, you’ll see that the determinant, trace, and eigenvalues of A and B (which is P inverse A P) remain identical. This is a powerful concept because it means we can transform a matrix into a simpler form to study these properties without losing the essential information about the underlying linear transformation.

What is a Diagonalizable Matrix?

A square matrix \(A\) is diagonalizable if it is similar to some diagonal matrix \(D\).

That is, if there exists an invertible matrix \(P\) such that:

\[ D = P^{-1}AP \]

In this case, the matrix \(P\) is said to diagonalize \(A\).

Why Diagonal?

Diagonal matrices are the simplest form of matrices. Their properties are immediately apparent:

• Eigenvalues are the diagonal entries.
• Determinant is the product of diagonal entries.
• Powers are simply powers of the diagonal entries (e.g., \(D^k\) has \(\lambda_i^k\) on its diagonal).

The Goal:

Find \(P\) such that \(P^{-1}AP = D\).

This means we are looking for a basis (formed by the columns of \(P\)) where the linear transformation represented by \(A\) acts simply by scaling each basis vector.

The scaling factors are the eigenvalues on the diagonal of \(D\).

Note

Diagonalization provides a “canonical form” for certain matrices, revealing their intrinsic properties.

The term “diagonalizable” literally means “can be made diagonal”. This transformation to a diagonal matrix is incredibly useful. Imagine trying to understand the effect of applying a complex linear transformation many times. If the matrix is diagonal, it’s trivial. If it’s not, it’s much harder. Diagonalization gives us a way to convert the complex case into the trivial case, at least conceptually.

Theorem 5.2.1: The Diagonalization Condition

This theorem provides the fundamental condition for a matrix to be diagonalizable.

Theorem 5.2.1

If \(A\) is an \(n \times n\) matrix, the following statements are equivalent:

1. \(A\) is diagonalizable.
2. \(A\) has \(n\) linearly independent eigenvectors.

Key Insight:

• If \(A\) has \(n\) linearly independent eigenvectors, these vectors can form the columns of the matrix \(P\).
• The diagonal entries of \(D\) will be the eigenvalues corresponding to these eigenvectors.

So, to diagonalize \(A\), we need to find its eigenvalues and a set of \(n\) linearly independent eigenvectors.

Proof Intuition:

From \(P^{-1}AP = D\), we can write \(AP = PD\).

If \(P = [\mathbf{p}_1 \ \mathbf{p}_2 \ \dots \ \mathbf{p}_n]\) and \(D = \operatorname{diag}(\lambda_1, \lambda_2, \dots, \lambda_n)\), then:

\[ A[\mathbf{p}_1 \ \dots \ \mathbf{p}_n] = [\lambda_1\mathbf{p}_1 \ \dots \ \lambda_n\mathbf{p}_n] \]

This implies \(A\mathbf{p}_i = \lambda_i\mathbf{p}_i\) for each column \(\mathbf{p}_i\).

Thus, the columns of \(P\) are eigenvectors, and the diagonal entries of \(D\) are their corresponding eigenvalues.

Since \(P\) must be invertible, its columns (\(\mathbf{p}_i\)) must be linearly independent.

This theorem is the cornerstone of diagonalization. It directly links the abstract concept of similarity to a concrete property of a matrix: its eigenvectors. The crucial part is “n linearly independent eigenvectors”. This is not always guaranteed, as we’ll see, and it’s what determines if a matrix is diagonalizable. The proof intuition shows how \(P\) is constructed from these eigenvectors and \(D\) from their eigenvalues.

When is a Matrix Diagonalizable? (Theorem 5.2.2)

While Theorem 5.2.1 gives the necessary and sufficient condition, Theorem 5.2.2 provides a simpler, often quicker, way to check for diagonalizability in many cases.

Theorem 5.2.2

1. If \(\lambda_1, \ldots, \lambda_k\) are distinct eigenvalues of a matrix \(A\), and if \(\mathbf{v}_1, \ldots, \mathbf{v}_k\) are corresponding eigenvectors, then \(\{\mathbf{v}_1, \ldots, \mathbf{v}_k\}\) is a linearly independent set.

2. An \(n \times n\) matrix with \(n\) distinct eigenvalues is diagonalizable.

Implication of (b):

If an \(n \times n\) matrix has \(n\) different eigenvalues, then it is automatically diagonalizable!

You don’t even need to find the eigenvectors to know it’s diagonalizable.

This is a very common scenario in many ECE applications.

Example:

Consider a \(3 \times 3\) matrix \(A\) with eigenvalues \(\lambda_1=1\), \(\lambda_2=2\), \(\lambda_3=3\).

Since all three eigenvalues are distinct, \(A\) is guaranteed to be diagonalizable.

\[ A = \left[ \begin{array}{ccc}0 & 1 & 0 \\ 0 & 0 & 1 \\ 4 & -17 & 8 \end{array} \right] \]

This matrix has three distinct eigenvalues: \(\lambda = 4\), \(\lambda = 2 + \sqrt{3}\), and \(\lambda = 2 - \sqrt{3}\).

Therefore, \(A\) is diagonalizable.

Theorem 5.2.2 (b) is a very practical result. If you can quickly find that an \(n \times n\) matrix has \(n\) distinct eigenvalues, you immediately know it’s diagonalizable, and you don’t need to go through the potentially tedious process of finding eigenvectors just to check for diagonalizability. This is often the first check performed.

Procedure for Diagonalizing an \(n \times n\) Matrix

If a matrix \(A\) is diagonalizable, here’s how to find \(P\) and \(D\):

Step 1: Find Eigenvalues

Calculate the characteristic equation \(\operatorname{det}(\lambda I - A) = 0\) and solve for the eigenvalues \(\lambda_1, \lambda_2, \ldots, \lambda_n\).

Step 2: Find Basis for Each Eigenspace

For each eigenvalue \(\lambda_i\), find a basis for the eigenspace \(E_{\lambda_i} = \operatorname{null}(\lambda_i I - A)\).

The goal is to find \(n\) linearly independent eigenvectors in total.

Procedure for Diagonalizing an \(n \times n\) Matrix

Step 3: Construct \(P\) and \(D\)

• If you found \(n\) linearly independent eigenvectors \(\mathbf{p}_1, \mathbf{p}_2, \ldots, \mathbf{p}_n\), then \(A\) is diagonalizable.
• Form \(P = [\mathbf{p}_1 \ \mathbf{p}_2 \ \dots \ \mathbf{p}_n]\).
• Form \(D = \operatorname{diag}(\lambda_1, \lambda_2, \dots, \lambda_n)\), where each \(\lambda_i\) corresponds to \(\mathbf{p}_i\).

Step 4: Verify

Check that \(P^{-1}AP = D\).

This procedure is a direct application of Theorem 5.2.1. The most critical part is Step 2: ensuring you can find n linearly independent eigenvectors. If the total count of basis vectors for all eigenspaces is less than \(n\), the matrix is not diagonalizable. The order of eigenvectors in \(P\) determines the order of eigenvalues in \(D\).

Example 1: Finding a Matrix \(P\) That Diagonalizes \(A\)

Let’s diagonalize the matrix:

\[ A={\left[\begin{array}{l l l}{0}&{0}&{-2}\\ {1}&{2}&{1}\\ {1}&{0}&{3}\end{array}\right]} \]

Step 1 & 2: Eigenvalues & Eigenvectors

The characteristic equation is \((\lambda -1)(\lambda -2)^{2} = 0\). Eigenvalues are \(\lambda=1\) and \(\lambda=2\) (with algebraic multiplicity 2).

Bases for eigenspaces:

• For \(\lambda=2\): \(\mathbf{p}_{1}={\left[\begin{array}{l}{-1}\\ {0}\\ {1}\end{array}\right]}\), \(\mathbf{p}_{2}={\left[\begin{array}{l}{0}\\ {1}\\ {0}\end{array}\right]}\)
• For \(\lambda=1\): \(\mathbf{p}_{3}={\left[\begin{array}{l}{-2}\\ {1}\\ {1}\end{array}\right]}\)

We have three linearly independent eigenvectors for a \(3 \times 3\) matrix. Thus, \(A\) is diagonalizable!

Example 1: Finding a Matrix \(P\) That Diagonalizes \(A\)

Step 3: Construct \(P\) and \(D\)

Form \(P\) using these eigenvectors as columns:

\[ P={\left[\begin{array}{l l l}{-1}&{0}&{-2}\\ {0}&{1}&{1}\\ {1}&{0}&{1}\end{array}\right]} \]

Form \(D\) with corresponding eigenvalues on the diagonal:

\[ D={\left[\begin{array}{l l l}{2}&{0}&{0}\\ {0}&{2}&{0}\\ {0}&{0}&{1}\end{array}\right]} \]

Step 4: Verification (Interactive)

Let’s verify \(P^{-1}AP = D\) using Python.

Example 1: Finding a Matrix \(P\) That Diagonalizes \(A\)

This example walks through the full process. Notice how the order of eigenvalues in D corresponds to the order of their respective eigenvectors in P. If we were to reorder the columns of P, the diagonal entries of D would also reorder accordingly. The Python code provides a quick verification of the result.

Example 2: A Matrix That Is Not Diagonalizable

Not all matrices are diagonalizable. Consider this matrix:

\[ A = \left[ \begin{array}{rrr} 1 & 0 & 0 \\ 1 & 2 & 0 \\ -3 & 5 & 2 \end{array} \right] \]

Step 1: Find Eigenvalues

The characteristic polynomial of \(A\) is \((\lambda -1)(\lambda -2)^2 = 0\).

The distinct eigenvalues are \(\lambda = 1\) and \(\lambda = 2\).

Example 2: A Matrix That Is Not Diagonalizable

Step 2: Find Basis for Each Eigenspace

• For \(\lambda = 1\): The eigenspace is 1-dimensional, with basis vector \(\mathbf{p}_1 = \left[ \begin{array}{c} \frac{1}{8} \\ -\frac{1}{8} \\ 1 \end{array} \right]\).
• For \(\lambda = 2\): The eigenspace is also 1-dimensional, with basis vector \(\mathbf{p}_2 = \left[ \begin{array}{c} 0 \\ 0 \\ 1 \end{array} \right]\).

Conclusion:

Since \(A\) is a \(3 \times 3\) matrix, but we found only two linearly independent eigenvectors in total, \(A\) is not diagonalizable.

We need \(n=3\) linearly independent eigenvectors to diagonalize a \(3 \times 3\) matrix.

Warning

The existence of repeated eigenvalues does not automatically mean a matrix is not diagonalizable, but it does mean we need to check the dimension of the eigenspaces carefully.

This example clearly shows a case where a matrix is not diagonalizable. Even though \(\lambda=2\) has an algebraic multiplicity of 2, its eigenspace only has a dimension of 1. This mismatch is key. We’ll formalize this concept with geometric and algebraic multiplicity soon.

Geometric and Algebraic Multiplicity

These concepts help us understand why some matrices with repeated eigenvalues are diagonalizable and others are not.

Algebraic Multiplicity (AM)

The number of times that \((\lambda - \lambda_0)\) appears as a factor in the characteristic polynomial of \(A\).

It’s the “count” of how many times an eigenvalue appears as a root.

Geometric Multiplicity (GM)

The dimension of the eigenspace corresponding to \(\lambda_0\). It’s the number of linearly independent eigenvectors associated with \(\lambda_0\).

Geometric and Algebraic Multiplicity

Theorem 5.2.4: Diagonalizability Criterion

1. For every eigenvalue \(\lambda_0\) of \(A\), its geometric multiplicity is less than or equal to its algebraic multiplicity:

   \[ \operatorname{GM}(\lambda_0) \le \operatorname{AM}(\lambda_0) \]

2. A is diagonalizable if and only if the geometric multiplicity of every eigenvalue is equal to its algebraic multiplicity.

   \[ \operatorname{GM}(\lambda_0) = \operatorname{AM}(\lambda_0) \quad \text{for all } \lambda_0 \]

This theorem is the most complete answer to the diagonalizability question. It tells us exactly when a matrix can be diagonalized, even in cases with repeated eigenvalues. The key is that every eigenspace must be “full” in the sense that its dimension matches the number of times its eigenvalue appears in the characteristic polynomial.

Interactive: Multiplicities Check

Let’s use the matrices from previous examples to see their multiplicities.

Matrix A (Diagonalizable):

\[ A={\left[\begin{array}{l l l}{0}&{0}&{-2}\\ {1}&{2}&{1}\\ {1}&{0}&{3}\end{array}\right]} \] Expected: \(\lambda=1\) (AM=1, GM=1), \(\lambda=2\) (AM=2, GM=2)

Matrix B (Not Diagonalizable):

\[ B = \left[ \begin{array}{rrr} 1 & 0 & 0 \\ 1 & 2 & 0 \\ -3 & 5 & 2 \end{array} \right] \]

Expected: \(\lambda=1\) (AM=1, GM=1), \(\lambda=2\) (AM=2, GM=1)

Interactive: Multiplicities Check

Choose a Matrix:

viewof matrix_choice = Inputs.select(["Matrix A (Diagonalizable)", "Matrix B (Not Diagonalizable)"], {label: "Select Matrix"});

This interactive block demonstrates Theorem 5.2.4. You can select between the two example matrices. The Python code calculates the algebraic multiplicity (by counting repeated eigenvalues) and the geometric multiplicity (by finding the nullity of \((\lambda I - A)\)). Observe how for the diagonalizable matrix, GM equals AM for all eigenvalues, whereas for the non-diagonalizable matrix, GM is less than AM for \(\lambda=2\).

ECE Application: Decoupling Systems

Many ECE systems are described by coupled differential equations, often represented in state-space form.

Diagonalization allows us to “decouple” these systems.

State-Space Representation:

A linear time-invariant system can be described by: \[ \dot{\mathbf{x}}(t) = A\mathbf{x}(t) + B\mathbf{u}(t) \] where \(\mathbf{x}\) is the state vector, \(\mathbf{u}\) is the input vector, and \(A\) is the system matrix.

If \(A\) is diagonalizable, we can define a new state vector \(\mathbf{z}(t) = P^{-1}\mathbf{x}(t)\).

Then \(\mathbf{x}(t) = P\mathbf{z}(t)\), and substituting into the state equation:

\[ P\dot{\mathbf{z}}(t) = AP\mathbf{z}(t) + B\mathbf{u}(t) \] \[ \dot{\mathbf{z}}(t) = P^{-1}AP\mathbf{z}(t) + P^{-1}B\mathbf{u}(t) \] \[ \dot{\mathbf{z}}(t) = D\mathbf{z}(t) + \tilde{B}\mathbf{u}(t) \] where \(D = P^{-1}AP\) and \(\tilde{B} = P^{-1}B\).

ECE Application: Decoupling Systems

Benefits of Decoupling:

The transformed system \(\dot{\mathbf{z}}(t) = D\mathbf{z}(t) + \tilde{B}\mathbf{u}(t)\) is much simpler:

• Since \(D\) is diagonal, each state variable \(z_i\) evolves independently. \[ \dot{z}_i(t) = \lambda_i z_i(t) + \tilde{b}_i \mathbf{u}(t) \]
• Easier Analysis: We can analyze each mode (eigenvalue) separately.
• Simpler Control Design: Control strategies can be designed for each independent mode.
• Modal Analysis: Eigenvalues \(\lambda_i\) represent the system’s natural modes (e.g., natural frequencies, damping ratios), and eigenvectors are the mode shapes.

ECE Application: Decoupling Systems

Diagram: Coupled vs. Decoupled Systems

This is a critical application in ECE. In control systems, for example, we often have systems where the behavior of one component affects all others. Diagonalization allows us to transform this complex, coupled system into an equivalent, simpler system where each component (or “mode”) evolves independently. This makes it much easier to predict the system’s behavior, design controllers, or analyze stability. The eigenvalues tell us how fast these modes decay or grow, and the eigenvectors tell us the “shape” of these modes.

Computing Powers of a Matrix (\(A^k\))

Computing high powers of a matrix \(A^k\) directly can be computationally intensive.

Diagonalization offers a highly efficient method.

The Formula:

If \(A\) is diagonalizable, \(P^{-1}AP = D\), which means \(A = PDP^{-1}\). Then, for any positive integer \(k\):

\[ A^k = (PDP^{-1})^k = (PDP^{-1})(PDP^{-1})\dots(PDP^{-1}) \] Since \(P^{-1}P = I\), intermediate terms cancel out: \[ A^k = PD (P^{-1}P) D (P^{-1}P) \dots (P^{-1}P) DP^{-1} = PD^kP^{-1} \] So, \[ A^k = P D^k P^{-1} = P \left[ \begin{array}{l l l l}{\lambda_{1}^{k}} & 0 & \dots & 0\\ 0 & {\lambda_{2}^{k}} & \dots & 0\\ \vdots & \vdots & & \vdots \\ 0 & 0 & \dots & {\lambda_{n}^{k}} \end{array} \right] P^{-1} \]

Computing Powers of a Matrix (\(A^k\))

Why is this efficient?

• Raising a diagonal matrix \(D\) to the power \(k\) is trivial: just raise each diagonal entry to the power \(k\).
• This avoids \(k-1\) matrix multiplications of \(A\) with itself, which is very costly for large \(k\).

Tip

This technique is vital in areas like Markov chains, difference equations, and simulating long-term system behavior.

Calculating \(A^k\) directly for large \(k\) is extremely inefficient and prone to numerical errors. The diagonalization method is a game-changer. Once you’ve found \(P\) and \(D\) (which only needs to be done once), calculating any power \(A^k\) becomes just two matrix multiplications and a simple element-wise exponentiation. This is used extensively in simulations and predictive modeling.

Example 6: Powers of a Matrix (Interactive)

Let’s compute \(A^k\) for the matrix from Example 1 using diagonalization.

\[ A={\left[\begin{array}{l l l}{0}&{0}&{-2}\\ {1}&{2}&{1}\\ {1}&{0}&{3}\end{array}\right]} \]

We found \(P\) and \(D\): \[ P={\left[\begin{array}{l l l}{-1}&{0}&{-2}\\ {0}&{1}&{1}\\ {1}&{0}&{1}\end{array}\right]},\quad D={\left[\begin{array}{l l l}{2}&{0}&{0}\\ {0}&{2}&{0}\\ {0}&{0}&{1}\end{array}\right]} \]

Example 6: Powers of a Matrix (Interactive)

Choose Power \(k\):

viewof k_power = Inputs.range([1, 20], {value: 3, step: 1, label: "Power k"});

Computed \(A^k\):

Here, you can dynamically change the power ‘k’ and see how \(A^k\) is computed using the diagonalization formula. Notice how the values can grow very quickly. For small ‘k’, we also show the direct computation for comparison, confirming the results. This clearly illustrates the power and efficiency of diagonalization for matrix powers.

Summary & Key Takeaways

What We Covered:

• Similarity Transformations: Changing basis while preserving core properties.
• Diagonalizable Matrices: Matrices similar to a diagonal matrix (\(D=P^{-1}AP\)).
• Key Condition: A matrix is diagonalizable iff it has \(n\) linearly independent eigenvectors.
• Shortcut: \(n\) distinct eigenvalues guarantees diagonalizability.
• Geometric vs. Algebraic Multiplicity: GM must equal AM for all eigenvalues for diagonalizability.

Summary & Key Takeaways

Why it Matters for ECE:

• System Decoupling: Simplifies analysis and control of complex, coupled systems (e.g., state-space models).
• Modal Analysis: Eigenvalues/eigenvectors reveal natural modes of behavior in circuits, signals, and control systems.
• Efficient Computation: Drastically speeds up calculations of matrix powers, essential for simulations and long-term predictions.

Note

Diagonalization is a fundamental tool for understanding and manipulating linear systems in all areas of ECE.

To recap, diagonalization is not just a mathematical curiosity; it’s a practical and indispensable tool in ECE. It allows us to break down complex systems into simpler, independent components, making analysis, design, and prediction much more manageable. Understanding when and how to diagonalize a matrix is a key skill for any ECE student.