Linear Algebra

5.1 Eigenvalues & Eigenvectors

Imron Rosyadi

Introduction: What are Eigenvalues & Eigenvectors?

Definition 1: Eigenvector and Eigenvalue

If \(A\) is an \(n \times n\) matrix, a nonzero vector \(\mathbf{x}\) in \(R^{n}\) is an eigenvector of \(A\) if:

\[ A\mathbf{x} = \lambda \mathbf{x} \]

for some scalar \(\lambda\).

The scalar \(\lambda\) is called an eigenvalue of \(A\).

\(\mathbf{x}\) is said to be an eigenvector corresponding to \(\lambda\).

Note

Eigenvectors are special vectors whose direction remains unchanged (or reversed) after a linear transformation by matrix \(A\).

Eigenvalues are the scaling factors by which these eigenvectors are stretched or compressed.

Introduction: What are Eigenvalues & Eigenvectors?

Key Idea:

Most vectors change both direction and magnitude when multiplied by a matrix \(A\).

Eigenvectors are unique: they only change magnitude (and possibly orientation by 180 degrees)

• \(A\) acts on \(x\)
• \(x\) is scaled by \(\lambda\)

Why \(x\) must be nonzero?

\(A\mathbf{0} = \lambda \mathbf{0}\) holds for any matrix \(A\) and any scalar \(\lambda\), which isn’t useful.

Welcome everyone to this session on Eigenvalues and Eigenvectors. These concepts are fundamental in Linear Algebra and have profound implications across various fields, especially in Electrical and Computer Engineering.

At its core, an eigenvector is a special vector that, when a linear transformation is applied to it (represented by a matrix A), only gets scaled, not rotated or flipped in a complex way. The scaling factor is what we call the eigenvalue.

Think of it like this: if you push a button on a system, most things might move in complicated ways. But an eigenvector represents a specific mode or direction of response where the system simply scales its output along that same direction. This simplicity is incredibly powerful for analysis.

Geometric Intuition

When a matrix \(A\) multiplies a vector \(\mathbf{x}\):

• Typically, \(A\mathbf{x}\) has a different direction and magnitude than \(\mathbf{x}\).
• For an eigenvector \(\mathbf{x}\), the direction of \(A\mathbf{x}\) is the same as \(\mathbf{x}\) (or exactly opposite if \(\lambda\) is negative).

The operation \(A\mathbf{x} = \lambda \mathbf{x}\) means \(\mathbf{x}\) is:

• Stretched by factor \(\lambda\) if \(\lambda > 1\).
• Compressed by factor \(\lambda\) if \(0 < \lambda < 1\).
• Reversed and stretched/compressed if \(\lambda < 0\).

This helps us understand the fundamental ‘modes’ or ‘directions’ of a linear transformation.

Geometric Intuition

Figure 5.1.1: \(A\mathbf{x}\) is a scalar multiple of \(\mathbf{x}\).

Figure 5.1.2: Example of stretching.

Let’s visualize this. Imagine a transformation in 2D space. If you take a random vector and apply the transformation, its new position will likely be in a completely different direction.

But for an eigenvector, it stays on the same line through the origin. It just moves further along that line, or closer to the origin, or even flips to the opposite side while still staying on that line.

The images here illustrate this: Figure 5.1.1 shows the general concept of \(A\mathbf{x}\) being parallel to \(\mathbf{x}\). Figure 5.1.2 gives a concrete example where the vector \(\mathbf{x}\) is stretched by a factor of 3, meaning \(\lambda=3\).

Example: Eigenvector of a \(2 \times 2\) Matrix

Problem: Verify if \(\mathbf{x} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}\) is an eigenvector of \(A = \begin{bmatrix} 3 & 0 \\ 8 & -1 \end{bmatrix}\) and find its corresponding eigenvalue.

Solution:

We compute \(A\mathbf{x}\):

\[ A\mathbf{x} = \begin{bmatrix} 3 & 0 \\ 8 & -1 \end{bmatrix} \begin{bmatrix} 1 \\ 2 \end{bmatrix} = \begin{bmatrix} (3)(1) + (0)(2) \\ (8)(1) + (-1)(2) \end{bmatrix} = \begin{bmatrix} 3 \\ 6 \end{bmatrix} \]

Now, we check if \(\begin{bmatrix} 3 \\ 6 \end{bmatrix}\) is a scalar multiple of \(\mathbf{x} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}\):

\[ \begin{bmatrix} 3 \\ 6 \end{bmatrix} = 3 \begin{bmatrix} 1 \\ 2 \end{bmatrix} = 3\mathbf{x} \]

Yes, it is!

So, \(\mathbf{x} = \begin{bmatrix} 1 \\ 2 \end{bmatrix}\) is an eigenvector of \(A\), and its corresponding eigenvalue is \(\lambda = 3\).

Important

This geometric transformation (stretching by a factor of 3) is exactly what we saw in Figure 5.1.2.

Example: Eigenvector of a \(2 \times 2\) Matrix

Quick Check:

Does \(A\mathbf{x} = \lambda \mathbf{x}\) hold?

\(\begin{bmatrix} 3 \\ 6 \end{bmatrix} = 3 \begin{bmatrix} 1 \\ 2 \end{bmatrix}\)

This confirms our result.

This example demonstrates the definition in practice. We take a given matrix and a vector, perform the matrix-vector multiplication, and then observe the result. If the result is simply a scaled version of the original vector, then we’ve found an eigenvector and its eigenvalue. Here, the vector (1,2) is stretched to (3,6), which is 3 times the original vector. This means lambda is 3.

Interactive Vector Transformation

Explore how a 2x2 matrix transforms a vector.

Adjust the matrix entries and vector components to observe the output.

Can you find an eigenvector?

viewof a_val = Inputs.range([-5, 5], {value: 3, step: 1, label: "A[0,0]"});
viewof b_val = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[0,1]"});
viewof c_val = Inputs.range([-5, 5], {value: 8, step: 1, label: "A[1,0]"});
viewof d_val = Inputs.range([-5, 5], {value: -1, step: 1, label: "A[1,1]"});

viewof x1_val = Inputs.range([-5, 5], {value: 1, step: 0.1, label: "x[0]"});
viewof x2_val = Inputs.range([-5, 5], {value: 2, step: 0.1, label: "x[1]"});

This interactive plot allows you to dynamically change the matrix elements and the components of vector X.

Observe how the blue vector (X) is transformed into the red dashed vector (AX). Most of the time, AX will point in a completely different direction.

Your challenge is to find combinations of A and X where AX lies exactly along the same line as X. When you find such a case, the plot will indicate “Eigenvector found!” and display the corresponding eigenvalue (lambda).

Try to recreate the previous example: A = [[3,0],[8,-1]] and X = [1,2]. You should see lambda = 3. Also, try A = [[3,0],[8,-1]] and X = [-3/2, 1] (you might need to adjust the range for X if it’s outside the slider). What lambda do you get?

Finding Eigenvalues: The Characteristic Equation

To find eigenvalues \(\lambda\) for a matrix \(A\):

1. Start with the definition: \(A\mathbf{x} = \lambda \mathbf{x}\)
2. Rewrite: \(A\mathbf{x} = \lambda I\mathbf{x}\) (where \(I\) is the identity matrix)
3. Rearrange: \((\lambda I - A)\mathbf{x} = \mathbf{0}\)

For this equation to have a nonzero solution \(\mathbf{x}\) (an eigenvector), the matrix \((\lambda I - A)\) must be singular.

This means its determinant must be zero.

Theorem 5.1.1: Characteristic Equation

If \(A\) is an \(n \times n\) matrix, then \(\lambda\) is an eigenvalue of \(A\) if and only if it satisfies the equation:

\[ \operatorname{det}(\lambda I - A) = 0 \tag{1} \]

This is called the characteristic equation of \(A\).

Process Overview:

The expression \(\operatorname{det}(\lambda I - A)\) will be a polynomial in \(\lambda\), called the characteristic polynomial of \(A\).

For an \(n \times n\) matrix, the characteristic polynomial will be of degree \(n\), meaning it can have at most \(n\) distinct eigenvalues.

Now that we understand what eigenvectors and eigenvalues are, the next logical step is to learn how to find them for any given matrix.

The key insight comes from rewriting the eigenvalue definition. By introducing the identity matrix I, we can factor out the vector x, leading to the homogeneous system (lambda I - A)x = 0.

For this system to have non-trivial solutions (which eigenvectors must be), the matrix (lambda I - A) cannot be invertible. A non-invertible matrix has a determinant of zero. This leads us directly to the characteristic equation: det(lambda I - A) = 0.

Solving this polynomial equation for lambda gives us all the eigenvalues of the matrix A. The diagram illustrates this step-by-step process.

Example: Finding Eigenvalues for a \(2 \times 2\) Matrix

Problem: Use the characteristic equation to find all eigenvalues of \(A = \begin{bmatrix} 3 & 0 \\ 8 & -1 \end{bmatrix}\).

Solution:

1. Form \(\lambda I - A\): \[ \lambda I - A = \lambda \begin{bmatrix} 1 & 0 \\ 0 & 1 \end{bmatrix} - \begin{bmatrix} 3 & 0 \\ 8 & -1 \end{bmatrix} = \begin{bmatrix} \lambda & 0 \\ 0 & \lambda \end{bmatrix} - \begin{bmatrix} 3 & 0 \\ 8 & -1 \end{bmatrix} = \begin{bmatrix} \lambda - 3 & 0 \\ -8 & \lambda + 1 \end{bmatrix} \]

2. Calculate \(\operatorname{det}(\lambda I - A)\): \[ \operatorname{det}(\lambda I - A) = \left| \begin{array}{cc}\lambda -3 & 0 \\ -8 & \lambda +1 \end{array} \right| = (\lambda - 3)(\lambda + 1) - (0)(-8) = (\lambda - 3)(\lambda + 1) \]

3. Set \(\operatorname{det}(\lambda I - A) = 0\): \[ (\lambda - 3)(\lambda + 1) = 0 \]

4. Solve for \(\lambda\): This equation gives us two solutions:

   \(\lambda - 3 = 0 \implies \lambda = 3\)

   \(\lambda + 1 = 0 \implies \lambda = -1\)

Thus, the eigenvalues of \(A\) are \(\lambda = 3\) and \(\lambda = -1\).

Tip

The characteristic polynomial for this matrix is \(p(\lambda) = (\lambda - 3)(\lambda + 1) = \lambda^2 - 2\lambda - 3\).

Let’s apply the characteristic equation method to the matrix from our earlier example.

First, we construct the matrix (lambda I - A). Remember, this involves subtracting A from a diagonal matrix with lambda on its main diagonal.

Next, we compute the determinant of this new matrix. For a 2x2 matrix, this is straightforward: (ad - bc). This calculation yields a polynomial in lambda.

Finally, we set this characteristic polynomial equal to zero and solve for lambda. For a quadratic equation like this, factoring or the quadratic formula works. We find the eigenvalues are 3 and -1. This confirms the eigenvalue of 3 we observed earlier, and also reveals a second eigenvalue.

Interactive Characteristic Polynomial & Eigenvalue Finder

Input the elements of a \(2 \times 2\) or \(3 \times 3\) matrix.

The tool will compute its characteristic polynomial and find its eigenvalues.

viewof matrix_dim = Inputs.radio(["2x2", "3x3"], {value: "2x2", label: "Matrix Dimension"});

// 2x2 inputs
viewof a22_00 = Inputs.range([-5, 5], {value: 3, step: 1, label: "A[0,0]"});
viewof a22_01 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[0,1]"});
viewof a22_10 = Inputs.range([-5, 5], {value: 8, step: 1, label: "A[1,0]"});
viewof a22_11 = Inputs.range([-5, 5], {value: -1, step: 1, label: "A[1,1]"});

// 3x3 inputs
viewof a33_00 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[0,0]"});
viewof a33_01 = Inputs.range([-5, 5], {value: 1, step: 1, label: "A[0,1]"});
viewof a33_02 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[0,2]"});
viewof a33_10 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[1,0]"});
viewof a33_11 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[1,1]"});
viewof a33_12 = Inputs.range([-5, 5], {value: 1, step: 1, label: "A[1,2]"});
viewof a33_20 = Inputs.range([-5, 5], {value: 4, step: 1, label: "A[2,0]"});
viewof a33_21 = Inputs.range([-5, 5], {value: -17, step: 1, label: "A[2,1]"});
viewof a33_22 = Inputs.range([-5, 5], {value: 8, step: 1, label: "A[2,2]"});

This interactive tool is a powerful way to practice finding eigenvalues.

First, select whether you want a 2x2 or 3x3 matrix using the radio button. Then, adjust the sliders to define your matrix A.

The Python code will then calculate and display the characteristic polynomial, p(lambda), and its roots, which are the eigenvalues.

Try the previous 2x2 example: A = [[3,0],[8,-1]]. You should get lambda = 3 and -1. Now, try the 3x3 matrix from Example 3 (A = [[0,1,0],[0,0,1],[4,-17,8]]) to see its characteristic polynomial and eigenvalues. Note that the eigenvalues might appear as complex numbers even if their imaginary part is very close to zero due to floating-point arithmetic.

Example: Finding Eigenvalues for a \(3 \times 3\) Matrix

Problem: Find the eigenvalues of \(A = \left[ \begin{array}{ccc}0 & 1 & 0 \\ 0 & 0 & 1 \\ 4 & -17 & 8 \end{array} \right]\).

Solution:

1. Form \(\lambda I - A\) and calculate its determinant: \[ \operatorname{det}(\lambda I - A) = \operatorname{det}\left[ \begin{array}{ccc}\lambda & -1 & 0 \\ 0 & \lambda & -1 \\ -4 & 17 & \lambda -8 \end{array} \right] \] Expanding the determinant (e.g., along the first row) yields the characteristic polynomial: \[ p(\lambda) = \lambda^{3} - 8\lambda^{2} + 17\lambda -4 \]

Example: Finding Eigenvalues for a \(3 \times 3\) Matrix

2. Set \(p(\lambda) = 0\) and solve:

   We need to solve \(\lambda^{3} - 8\lambda^{2} + 17\lambda -4 = 0\).

   • Hint: Integer solutions must be divisors of the constant term (-4), i.e., \(\pm 1, \pm 2, \pm 4\).
   • Testing \(\lambda = 4\): \((4)^3 - 8(4)^2 + 17(4) - 4 = 64 - 128 + 68 - 4 = 0\).
   • So, \(\lambda = 4\) is an eigenvalue, and \((\lambda - 4)\) is a factor.
   • Dividing \((\lambda^3 - 8\lambda^2 + 17\lambda - 4)\) by \((\lambda - 4)\) gives \((\lambda^2 - 4\lambda + 1)\).
   • The equation becomes: \((\lambda - 4)(\lambda^{2} - 4\lambda + 1) = 0\).

3. Solve the quadratic factor:

   Using the quadratic formula for \(\lambda^{2} - 4\lambda + 1 = 0\):

   \[ \lambda = \frac{-(-4) \pm \sqrt{(-4)^2 - 4(1)(1)}}{2(1)} = \frac{4 \pm \sqrt{16 - 4}}{2} = \frac{4 \pm \sqrt{12}}{2} = \frac{4 \pm 2\sqrt{3}}{2} = 2 \pm \sqrt{3} \]

Thus, the eigenvalues of \(A\) are:

\[ \lambda = 4,\quad \lambda = 2 + \sqrt{3},\quad \mathrm{and}\quad \lambda = 2 - \sqrt{3} \]

Finding eigenvalues for 3x3 matrices involves solving a cubic characteristic equation. While the interactive tool can do this numerically, it’s important to understand the manual process.

For integer coefficients, we can often find integer roots by testing divisors of the constant term. Here, testing lambda = 4 reveals it’s a root. This allows us to factor the cubic polynomial into a linear term and a quadratic term. The quadratic term can then be solved using the quadratic formula, yielding the remaining two eigenvalues, which in this case are irrational.

Eigenvalues of Triangular Matrices

Theorem 5.1.2:

If \(A\) is an \(n \times n\) triangular matrix (upper, lower, or diagonal), then the eigenvalues of \(A\) are simply the entries on the main diagonal of \(A\).

Why?

Recall that the determinant of a triangular matrix is the product of its diagonal entries.

For a triangular matrix \(A\), the matrix \((\lambda I - A)\) will also be triangular.

Example for a \(4 \times 4\) upper triangular matrix: \[ \operatorname{det}(\lambda I - A) = \operatorname{det}\left[ \begin{array}{cccc}{\lambda -a_{11}} & {-a_{12}} & {-a_{13}} & {-a_{14}}\\ 0 & {\lambda -a_{22}} & {-a_{23}} & {-a_{24}}\\ 0 & 0 & {\lambda -a_{33}} & {-a_{34}}\\ 0 & 0 & 0 & {\lambda -a_{44}} \end{array} \right] \] \[ = (\lambda -a_{11})(\lambda -a_{22})(\lambda -a_{33})(\lambda -a_{44}) \] Setting this to zero immediately gives \(\lambda = a_{11}, a_{22}, a_{33}, a_{44}\).

Eigenvalues of Triangular Matrices

Example 4: Upper Triangular Matrix

For \(A = {\left[ \begin{array}{l l l l}{a_{11}} & {a_{12}} & {a_{13}} & {a_{14}}\\ 0 & {a_{22}} & {a_{23}} & {a_{24}}\\ 0 & 0 & {a_{33}} & {a_{34}}\\ 0 & 0 & 0 & {a_{44}} \end{array} \right]}\), the eigenvalues are \(\lambda = a_{11}, \lambda = a_{22}, \lambda = a_{33}, \lambda = a_{44}\).

Example 5: Lower Triangular Matrix

For \(A = \left[ \begin{array}{ccc}\frac{1}{2} & 0 & 0 \\ -1 & \frac{2}{3} & 0 \\ 5 & -8 & -\frac{1}{4} \end{array} \right]\), the eigenvalues are \(\lambda = \frac{1}{2}, \lambda = \frac{2}{3}, \lambda = - \frac{1}{4}\).

Note

This theorem provides a very quick way to find eigenvalues for certain types of matrices, common in system analysis (e.g., state-space representations that are upper/lower triangular).

This theorem is a fantastic shortcut. For any triangular matrix—whether upper, lower, or even diagonal—you don’t need to compute the determinant and solve a polynomial. The eigenvalues are simply the entries on the main diagonal.

This is because when you form (lambda I - A), the resulting matrix is also triangular. And the determinant of a triangular matrix is just the product of its diagonal elements. Setting that product to zero immediately shows that each diagonal element (a_ii) must be an eigenvalue. This is a very useful property in ECE, especially when dealing with system matrices that can be put into triangular forms.

Finding Eigenvectors: Eigenspaces

Once you have found an eigenvalue \(\lambda\), you can find its corresponding eigenvectors \(\mathbf{x}\) by solving the homogeneous system:

\[ (\lambda I - A)\mathbf{x} = \mathbf{0} \]

Eigenspace:

The set of all solutions to this system is called the eigenspace of \(A\) corresponding to \(\lambda\).

• It’s a subspace of \(R^n\).
• It includes the zero vector (which is not an eigenvector by definition).
• We look for a basis for this eigenspace, which will consist of actual eigenvectors.

Interpretation: The eigenspace represents all vectors that are “transformed” along the same direction by \(A\), scaled by \(\lambda\).

Finding Eigenvectors: Eigenspaces

Process to Find Eigenspace Basis:

1. Find the eigenvalues \(\lambda\) (using the characteristic equation).
2. For each eigenvalue \(\lambda_k\):
   1. Form the matrix \((\lambda_k I - A)\).
   2. Solve the homogeneous system \((\lambda_k I - A)\mathbf{x} = \mathbf{0}\).
   3. Express the general solution in parametric vector form.
   4. The vectors that form the basis for this general solution are the basis vectors for the eigenspace corresponding to \(\lambda_k\).

Caution

Remember to exclude the zero vector when referring to actual eigenvectors, even though it’s part of the eigenspace.

After finding the eigenvalues, the next step is to find the corresponding eigenvectors. This involves going back to the equation (lambda I - A)x = 0.

For each eigenvalue we found, we substitute it back into this equation and solve for x. This is essentially finding the null space of the matrix (lambda I - A).

The set of all solutions, including the zero vector, forms what’s called the eigenspace. We’re usually interested in finding a basis for this eigenspace, as these basis vectors themselves are eigenvectors and can generate all other eigenvectors for that particular lambda.

The process is standard: set up the augmented matrix for the homogeneous system, perform Gaussian elimination to get to reduced row echelon form, and then write out the general solution in terms of free variables. Each free variable will correspond to a basis vector for the eigenspace.

Example: Eigenspaces for a \(2 \times 2\) Matrix

Problem: Find bases for the eigenspaces of \(A = \left[ \begin{array}{cc} - 1 & 3 \\ 2 & 0 \end{array} \right]\).

Solution:

From an earlier example (or using the characteristic equation), the eigenvalues are \(\lambda = 2\) and \(\lambda = -3\).

Case 1: Eigenspace for \(\lambda = 2\)

We solve \((\lambda I - A)\mathbf{x} = \mathbf{0}\) for \(\lambda = 2\): \[ \left[ \begin{array}{cc}2+1 & -3 \\ -2 & 2 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \end{array} \right] = \left[ \begin{array}{cc}3 & -3 \\ -2 & 2 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \end{array} \right] = \left[ \begin{array}{c}0 \\ 0 \end{array} \right] \] This system reduces to \(3x_1 - 3x_2 = 0 \implies x_1 = x_2\).

Let \(x_2 = t\), then \(x_1 = t\).

The general solution is \(\mathbf{x} = \begin{bmatrix} t \\ t \end{bmatrix} = t \begin{bmatrix} 1 \\ 1 \end{bmatrix}\).

A basis for the eigenspace corresponding to \(\lambda = 2\) is \(\left\{ \begin{bmatrix} 1 \\ 1 \end{bmatrix} \right\}\).

Example: Eigenspaces for a \(2 \times 2\) Matrix

Case 2: Eigenspace for \(\lambda = -3\)

We solve \((\lambda I - A)\mathbf{x} = \mathbf{0}\) for \(\lambda = -3\): \[ \left[ \begin{array}{cc}-3+1 & -3 \\ -2 & -3 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \end{array} \right] = \left[ \begin{array}{cc}-2 & -3 \\ -2 & -3 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \end{array} \right] = \left[ \begin{array}{c}0 \\ 0 \end{array} \right] \] This system reduces to \(-2x_1 - 3x_2 = 0 \implies x_1 = -\frac{3}{2}x_2\).

Let \(x_2 = t\), then \(x_1 = -\frac{3}{2}t\).

The general solution is \(\mathbf{x} = \begin{bmatrix} -\frac{3}{2}t \\ t \end{bmatrix} = t \begin{bmatrix} -\frac{3}{2} \\ 1 \end{bmatrix}\).

A basis for the eigenspace corresponding to \(\lambda = -3\) is \(\left\{ \begin{bmatrix} -\frac{3}{2} \\ 1 \end{bmatrix} \right\}\).

Example: Eigenspaces for a \(2 \times 2\) Matrix

Figure 5.1.3: Geometric effect of multiplication by A. \(L_1\) (for \(\lambda=2\)) and \(L_2\) (for \(\lambda=-3\)) are the eigenspaces.

Here, we take the eigenvalues we found earlier (2 and -3) and use them to find the corresponding eigenvectors.

For lambda = 2, we set up the system (2I - A)x = 0. Solving this system (which is equivalent to finding the null space of (2I-A)) gives us all vectors where x1 = x2. This forms a line in R2, and a basis vector for this line is (1,1).

Similarly, for lambda = -3, we find the null space of (-3I - A). This gives us another line in R2, with a basis vector (-3/2, 1).

The image vividly shows these two lines, L1 and L2. Any vector on L1, when multiplied by A, stays on L1 and is scaled by 2. Any vector on L2, when multiplied by A, stays on L2 and is scaled by -3 (meaning it flips direction and is stretched). This geometric interpretation is crucial for understanding.

Example: Eigenspaces for a \(3 \times 3\) Matrix

Problem: Find bases for the eigenspaces of \(A = \left[ \begin{array}{ccc}0 & 0 & -2 \\ 1 & 2 & 1 \\ 1 & 0 & 3 \end{array} \right]\).

Solution:

From an earlier example, the eigenvalues are \(\lambda = 1\) and \(\lambda = 2\) (with algebraic multiplicity 2).

Case 1: Eigenspace for \(\lambda = 2\)

Solve \((\lambda I - A)\mathbf{x} = \mathbf{0}\) for \(\lambda = 2\): \[ \left[ \begin{array}{ccc}2 & 0 & 2 \\ -1 & 0 & -1 \\ -1 & 0 & -1 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \\ x_{3} \end{array} \right] = \left[ \begin{array}{c}0 \\ 0 \\ 0 \end{array} \right] \] This system reduces to \(x_1 + x_3 = 0 \implies x_1 = -x_3\).

\(x_2\) is a free variable.

Let \(x_3 = s\) and \(x_2 = t\). Then \(x_1 = -s\).

The general solution is \(\mathbf{x} = \begin{bmatrix} -s \\ t \\ s \end{bmatrix} = s \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix} + t \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix}\).

A basis for the eigenspace corresponding to \(\lambda = 2\) is \(\left\{ \begin{bmatrix} -1 \\ 0 \\ 1 \end{bmatrix}, \begin{bmatrix} 0 \\ 1 \\ 0 \end{bmatrix} \right\}\).

Example: Eigenspaces for a \(3 \times 3\) Matrix

Case 2: Eigenspace for \(\lambda = 1\)

Solve \((\lambda I - A)\mathbf{x} = \mathbf{0}\) for \(\lambda = 1\): \[ \left[ \begin{array}{ccc}1 & 0 & 2 \\ -1 & -1 & -1 \\ -1 & 0 & -2 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \\ x_{3} \end{array} \right] = \left[ \begin{array}{c}0 \\ 0 \\ 0 \end{array} \right] \] Gaussian elimination on the augmented matrix:

\(\begin{bmatrix} 1 & 0 & 2 & | & 0 \\ -1 & -1 & -1 & | & 0 \\ -1 & 0 & -2 & | & 0 \end{bmatrix} \xrightarrow{R_2+R_1, R_3+R_1} \begin{bmatrix} 1 & 0 & 2 & | & 0 \\ 0 & -1 & 1 & | & 0 \\ 0 & 0 & 0 & | & 0 \end{bmatrix} \xrightarrow{-R_2} \begin{bmatrix} 1 & 0 & 2 & | & 0 \\ 0 & 1 & -1 & | & 0 \\ 0 & 0 & 0 & | & 0 \end{bmatrix}\)

This gives \(x_1 + 2x_3 = 0 \implies x_1 = -2x_3\) and \(x_2 - x_3 = 0 \implies x_2 = x_3\).

Let \(x_3 = s\). Then \(x_1 = -2s\) and \(x_2 = s\).

The general solution is \(\mathbf{x} = \begin{bmatrix} -2s \\ s \\ s \end{bmatrix} = s \begin{bmatrix} -2 \\ 1 \\ 1 \end{bmatrix}\).

A basis for the eigenspace corresponding to \(\lambda = 1\) is \(\left\{ \begin{bmatrix} -2 \\ 1 \\ 1 \end{bmatrix} \right\}\).

This 3x3 example shows that an eigenspace can have more than one basis vector. Here, for lambda = 2, we find two linearly independent basis vectors. This means the eigenspace for lambda = 2 is a 2-dimensional plane. For lambda = 1, we find a single basis vector, indicating a 1-dimensional line.

The number of basis vectors for an eigenspace is called its geometric multiplicity. It can be less than or equal to the algebraic multiplicity (the power of (lambda - eigenvalue) in the characteristic polynomial).

Interactive Eigenspace Basis Finder

Input a matrix and one of its eigenvalues.

The tool will find a basis for the corresponding eigenspace.

viewof es_matrix_dim = Inputs.radio(["2x2", "3x3"], {value: "2x2", label: "Matrix Dimension"});

// 2x2 inputs
viewof es_a22_00 = Inputs.range([-5, 5], {value: -1, step: 1, label: "A[0,0]"});
viewof es_a22_01 = Inputs.range([-5, 5], {value: 3, step: 1, label: "A[0,1]"});
viewof es_a22_10 = Inputs.range([-5, 5], {value: 2, step: 1, label: "A[1,0]"});
viewof es_a22_11 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[1,1]"});
viewof es_lambda22 = Inputs.range([-5, 5], {value: 2, step: 1, label: "Eigenvalue λ"});

// 3x3 inputs
viewof es_a33_00 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[0,0]"});
viewof es_a33_01 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[0,1]"});
viewof es_a33_02 = Inputs.range([-5, 5], {value: -2, step: 1, label: "A[0,2]"});
viewof es_a33_10 = Inputs.range([-5, 5], {value: 1, step: 1, label: "A[1,0]"});
viewof es_a33_11 = Inputs.range([-5, 5], {value: 2, step: 1, label: "A[1,1]"});
viewof es_a33_12 = Inputs.range([-5, 5], {value: 1, step: 1, label: "A[1,2]"});
viewof es_a33_20 = Inputs.range([-5, 5], {value: 1, step: 1, label: "A[2,0]"});
viewof es_a33_21 = Inputs.range([-5, 5], {value: 0, step: 1, label: "A[2,1]"});
viewof es_a33_22 = Inputs.range([-5, 5], {value: 3, step: 1, label: "A[2,2]"});
viewof es_lambda33 = Inputs.range([-5, 5], {value: 2, step: 1, label: "Eigenvalue λ"});

This interactive tool helps you compute the basis for an eigenspace.

First, select the matrix dimension and input the matrix A. Then, input a known eigenvalue lambda for that matrix.

The Python code will then form (lambda I - A) and compute its null space, which is the eigenspace. The output will be the basis vectors that span this eigenspace.

Try the 2x2 example: A = [[-1,3],[2,0]] with lambda = 2. You should get a basis vector like [1,1]. Then try lambda = -3, you should get a basis vector like [-1.5,1].

For the 3x3 example: A = [[0,0,-2],[1,2,1],[1,0,3]] with lambda = 2. You should get two basis vectors. Try lambda = 1, and you should get one basis vector.

Eigenvalues and Matrix Invertibility

Theorem 5.1.4:

A square matrix \(A\) is invertible if and only if \(\lambda = 0\) is not an eigenvalue of \(A\).

Proof Idea:

• If \(\lambda = 0\) is an eigenvalue, then \(A\mathbf{x} = 0\mathbf{x} = \mathbf{0}\) for some nonzero \(\mathbf{x}\).
• This means the homogeneous system \(A\mathbf{x} = \mathbf{0}\) has a non-trivial solution.
• If \(A\mathbf{x} = \mathbf{0}\) has a non-trivial solution, then \(A\) is not invertible.
• Conversely, if \(A\) is not invertible, then \(A\mathbf{x} = \mathbf{0}\) has a non-trivial solution \(\mathbf{x}\), implying \(A\mathbf{x} = 0\mathbf{x}\), so \(\lambda=0\) is an eigenvalue.

Eigenvalues and Matrix Invertibility

Example 8:

The matrix \(A = \left[ \begin{array}{ccc}0 & 0 & -2 \\ 1 & 2 & 1 \\ 1 & 0 & 3 \end{array} \right]\) has eigenvalues \(\lambda = 1\) and \(\lambda = 2\).

Since neither of these is zero, \(A\) is invertible.

(You can verify this by checking \(\operatorname{det}(A) \neq 0\)).

Theorem 5.1.5: Equivalent Statements

If \(A\) is an \(n \times n\) matrix, the following statements are equivalent:

1. \(A\) is invertible.
   …
   
2. \(\operatorname{det}(A) \neq 0\).
   …
   
3. \(\lambda = 0\) is not an eigenvalue of \(A\).

Important

This theorem adds a crucial link between eigenvalues and many other fundamental properties of a matrix, including its rank, nullity, and the nature of solutions to linear systems.

This theorem provides another powerful connection between eigenvalues and other core concepts in linear algebra. It states that a matrix is invertible if and only if zero is not one of its eigenvalues.

The proof is quite intuitive: if lambda equals zero is an eigenvalue, it means there’s a non-zero vector x such that Ax = 0x, which simplifies to Ax = 0. This is exactly the condition for a matrix to be non-invertible, as it implies the null space contains non-trivial vectors.

This adds a new item (u) to the long list of equivalent statements for an invertible matrix, which you’ve likely seen before. It’s a quick way to check if a matrix is singular or non-singular just by looking at its eigenvalues.

ECE Application: Linear Time-Invariant (LTI) Systems

Concept:

In ECE, Linear Time-Invariant (LTI) systems are fundamental (e.g., filters, amplifiers, control systems).

When a complex exponential signal, \(e^{j\omega t}\), is passed through an LTI system, the output is simply a scaled version of the input signal.

Mathematical Form:

If \(T\) is an LTI operator and the input is \(\mathbf{x}(t) = e^{j\omega t}\): \[ T(e^{j\omega t}) = H(j\omega) e^{j\omega t} \]

Here:

• \(e^{j\omega t}\) is an eigenfunction (continuous analog of eigenvector).
• \(H(j\omega)\) is the eigenvalue (the system’s frequency response).

This is exactly the definition \(T(\mathbf{x}) = \lambda \mathbf{x}\)!

ECE Application: Linear Time-Invariant (LTI) Systems

Diagram: LTI System as an Eigen-Operator

graph LR
    Input["Input Signal: $$e^{j\omega t}$$"] --> LTI_System("LTI System (T)");
    LTI_System --> Output["Output Signal: $$H(j\omega)e^{j\omega t}$$"];
    subgraph Properties
        H("Frequency Response $$H(j\omega)$$");
    end
    H -- "is the eigenvalue" --> LTI_System;
    Input -- "is the eigenfunction" --> LTI_System;

Significance:

• Fourier Analysis: Decomposes complex signals into sums of these eigenfunctions.
• Frequency Response: \(H(j\omega)\) tells us how the system amplifies/attenuates and shifts the phase of different frequency components.
• System Stability: Related to the location of eigenvalues in the complex plane.

This is where eigenvalues and eigenvectors truly shine in ECE. LTI systems are ubiquitous, from simple RC circuits to complex communication systems.

The remarkable property of LTI systems is that if you feed them a complex exponential signal (which forms the basis of Fourier analysis), the output is the exact same signal, just scaled by a complex number. This complex scaling factor is the eigenvalue, and the complex exponential itself is the eigenfunction.

This is a direct application of the eigenvalue-eigenvector concept, extended to functions. It’s why we use Fourier transforms to analyze systems in the frequency domain – because the complex exponentials are the “natural modes” or “eigenfunctions” of LTI systems. The eigenvalue H(j omega) tells us everything about how the system responds to that specific frequency.

Interactive LTI System Response (Frequency Domain)

Observe how an LTI system (simple filter) scales and shifts a sinusoidal input.

The output is a sinusoid of the same frequency, but with different amplitude and phase.

viewof input_amplitude = Inputs.range([0.1, 2.0], {value: 1, step: 0.1, label: "Input Amplitude"});
viewof input_frequency = Inputs.range([0.1, 5.0], {value: 1, step: 0.1, label: "Input Frequency (Hz)"});
viewof filter_gain = Inputs.range([0.1, 2.0], {value: 1.2, step: 0.1, label: "Filter Gain (Amplitude Scale)"});
viewof filter_phase = Inputs.range([-3.14, 3.14], {value: 0.5, step: 0.1, label: "Filter Phase Shift (rad)"});

This interactive plot helps visualize the LTI system concept.

You can adjust the amplitude and frequency of the input sine wave. Then, you can change the “filter gain” (which represents the magnitude of the eigenvalue H(j omega)) and the “filter phase shift” (which represents the argument of H(j omega)).

Notice that the output signal (red dashed line) always remains a sine wave of the same frequency as the input. Only its amplitude and phase are altered. This illustrates how sinusoidal (or complex exponential) signals are eigenfunctions of LTI systems. The system simply scales and shifts them.

Other ECE Applications

1. Control Systems:
   • Stability Analysis: Eigenvalues of the system matrix (A in \(\dot{\mathbf{x}} = A\mathbf{x}\)) determine system stability. If all eigenvalues have negative real parts, the system is stable.
   • Modal Analysis: Eigenvectors represent the “natural modes” of oscillation or decay of a system.
2. Image Processing & Computer Vision:
   • Principal Component Analysis (PCA): Used for dimensionality reduction and feature extraction. Eigenvectors of the covariance matrix represent the principal components (directions of maximum variance) in image data.
   • Image Compression: Retaining only eigenvectors corresponding to large eigenvalues for reconstruction.
3. Circuit Analysis:
   • Natural Frequencies: In RLC circuits, eigenvalues help determine the natural frequencies and damping factors of the circuit’s transient response.
   • Network Stability: Analyzing the stability of complex electrical networks.

Other ECE Applications

4. Quantum Computing & Physics:
   • Quantum States: Eigenvectors represent quantum states (e.g., energy levels).
   • Observables: Eigenvalues correspond to the possible measurable values of physical quantities (observables).
5. Vibration Analysis:
   • Mechanical Structures: Finding natural frequencies and modes of vibration in mechanical systems, crucial for design in aerospace and civil engineering. (Often involves ECE sensors and control).
6. Communication Systems:
   • MIMO Systems: Eigenvalue decomposition is used in Multiple-Input Multiple-Output (MIMO) wireless communication to optimize channel capacity and reduce interference.
   • Signal Detection: Designing optimal detectors based on signal and noise covariance matrices.

Beyond LTI systems, eigenvalues and eigenvectors permeate many other areas of ECE.

In control systems, eigenvalues are the go-to tool for stability analysis. Their real parts tell us if a system will grow, decay, or oscillate.

In image processing, PCA, a very common technique, relies entirely on eigenvectors to find the most significant directions of variation in data, enabling efficient compression and pattern recognition.

Even in basic circuit analysis, eigenvalues emerge when solving differential equations for RLC circuits, revealing their natural resonant behaviors.

And for those interested in quantum computing, eigenvalues and eigenvectors are the very language of quantum mechanics, describing states and measurable quantities. This list just scratches the surface of their utility in our field.

Summary & Key Takeaways

1. Core Definition:

• \(A\mathbf{x} = \lambda \mathbf{x}\)
• Eigenvectors (\(\mathbf{x}\)): Special directions unchanged by matrix transformation.
• Eigenvalues (\(\lambda\)): Scaling factors for eigenvectors.

2. Finding Eigenvalues:

• Solve the characteristic equation: \(\operatorname{det}(\lambda I - A) = 0\).
• Results in a polynomial whose roots are the eigenvalues.
• Triangular matrices: Eigenvalues are diagonal entries.

3. Finding Eigenvectors:

• For each \(\lambda\), solve the homogeneous system \((\lambda I - A)\mathbf{x} = \mathbf{0}\).
• The solution space is the eigenspace for \(\lambda\). Find a basis for this space.

Summary & Key Takeaways

4. Significance & ECE Applications:

• Geometric Interpretation: Understanding fundamental modes of transformation.
• Invertibility: \(A\) is invertible if and only if \(\lambda=0\) is not an eigenvalue.
• LTI Systems: Complex exponentials are eigenfunctions, frequency response \(H(j\omega)\) are eigenvalues.
• Control Systems: Stability analysis, modal decomposition.
• Image Processing: PCA, dimensionality reduction.
• Circuit Analysis: Natural frequencies.

Tip

Eigenvalues and eigenvectors help us understand the intrinsic behavior of systems, providing a simplified view of complex linear transformations.

To recap, eigenvalues and eigenvectors are fundamental concepts that reveal the inherent properties of linear transformations.

Eigenvectors are those special directions that remain invariant, only scaled by the eigenvalue. We find eigenvalues by solving the characteristic equation, which is the determinant of (lambda I - A) set to zero. Once we have the eigenvalues, we find the corresponding eigenvectors by solving the homogeneous system (lambda I - A)x = 0, which gives us the eigenspace.

These concepts are not just theoretical curiosities. In ECE, they are practical tools for analyzing system stability, understanding frequency responses in signal processing, performing data compression in image processing, and much more. They allow us to break down complex system behaviors into simpler, fundamental modes.