Linear Algebra

1.3 Matrices and Matrix Operations

Imron Rosyadi

Matrices and Matrix Operations

Beyond augmented matrices for linear systems, rectangular arrays of numbers are fundamental entities in their own right. In this section, we define matrices and their basic arithmetic operations.

We will cover:

• Matrix notation and terminology.
• Basic matrix operations: equality, addition, subtraction, scalar multiplication.
• The crucial operation of matrix multiplication.
• Special concepts: partitioned matrices, linear combinations, transpose, and trace.

Good morning, everyone! Today, we transition from using matrices primarily as tools for solving linear equations to viewing them as mathematical objects with their own rules of arithmetic. Matrices are incredibly versatile and appear in almost every branch of engineering, from representing signals and images to modeling control systems and electrical networks.

Understanding matrix operations is not just for theory; it’s essential for practical applications. When you’re trying to combine sensor data, apply transformations in computer graphics, or analyze complex circuit interactions, you’ll be performing matrix operations. We’ll start with basic definitions and then dive into the operations, especially matrix multiplication, which powers many transformations and computations in ECE.

Matrix Notation and Terminology

A matrix is a rectangular array of numbers. The numbers are called entries.

Example: Student study hours (Mon-Sun over 3 subjects) \[ \left[{\begin{array}{c c c c c c c}{2}&{3}&{2}&{4}&{1}&{4}&{2}\\ {0}&{3}&{1}&{4}&{3}&{2}&{2}\\ {4}&{1}&{3}&{1}&{0}&{0}&{2}\end{array}}\right] \]

• Size: Described by rows x columns (e.g., \(3 \times 7\)).
• Row Vector (Row Matrix): One row (e.g., \([2 \ 1 \ 0 \ -3]\)).
• Column Vector (Column Matrix): One column (e.g., \(\left[\begin{smallmatrix} 1 \\ 3 \\ 3 \end{smallmatrix}\right]\)).
• Square Matrix of order \(n\): \(n\) rows and \(n\) columns.
• Entry Notation: \(a_{ij}\) (or \((A)_{ij}\)) denotes the entry in row \(i\) and column \(j\).
• Main Diagonal: For a square matrix, entries \(a_{11}, a_{22}, \ldots, a_{nn}\).

\[ A = \left[ \begin{array}{cccc}a_{11} & a_{12} & \dots & a_{1n}\\ a_{21} & a_{22} & \dots & a_{2n}\\ \vdots & \vdots & & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{array} \right] \quad \text{general } m \times n \text{ matrix} \]

Let’s start with the basics of how we refer to matrices. A matrix is simply an organized collection of numbers, arranged in rows and columns. This structured format makes them ideal for representing various kinds of data in engineering. For example, a matrix could represent: * Pixel values in an image (rows \(\times\) columns of pixel intensity). * Sensor readings over time (rows for sensors, columns for time points). * System states in a dynamic model.

We describe its size as “rows by columns.” So, a \(3 \times 7\) matrix has three rows and seven columns. Special cases include row and column vectors, which are just matrices with a single row or column, respectively. Entries are identified by their row and column index, \(a_{ij}\), where \(i\) is the row and \(j\) is the column. This notation is super important for pinpointing individual elements. For a square matrix (same number of rows and columns), the elements running from top-left to bottom-right form the “main diagonal.” This diagonal has special significance in many matrix operations.

Matrix Equality, Addition, and Subtraction

Matrix Equality

Two matrices \(A\) and \(B\) are equal if:

1. They have the same size.
2. Their corresponding entries are equal (\(a_{ij} = b_{ij}\) for all \(i,j\)).

Example: \(A = \left[\begin{smallmatrix} 2 & 1 \\ 3 & x \end{smallmatrix}\right]\), \(B = \left[\begin{smallmatrix} 2 & 1 \\ 3 & 5 \end{smallmatrix}\right]\). \(A=B\) iff \(x=5\).

Matrix Addition and Subtraction

If \(A\) and \(B\) are matrices of the same size:

• Sum \(A+B\): Obtained by adding corresponding entries. \((A+B)_{ij} = a_{ij} + b_{ij}\).
• Difference \(A-B\): Obtained by subtracting corresponding entries. \((A-B)_{ij} = a_{ij} - b_{ij}\). Matrices of different sizes cannot be added or subtracted.

Much like scalars, matrices can be equal, added, and subtracted, but with specific rules. Equality is straightforward: they must be identical in terms of shape (same number of rows and columns) and content (every single entry must match). The example clearly shows this; if even one entry doesn’t match, the matrices are not equal.

Addition and Subtraction are elemental. They are defined only when the matrices have the exact same dimensions. You simply add or subtract the entries that are in the same position in both matrices. This is analogous to combining two sets of sensor data where each sensor corresponds to a specific location or type. For instance, if you have two snapshots of temperature readings from a grid of sensors at two different times, adding them might represent the total heat absorbed, or subtracting them would show the change in temperature at each point.

Interactive Example: Matrix Operations

Let’s demonstrate matrix addition and subtraction with Python:

Let’s use Python’s NumPy library to quickly perform these operations. NumPy is the standard for numerical computing in Python and is highly optimized for matrix operations.

As you can see from the output, both addition and subtraction are done “element-wise”. This means the corresponding entries are simply added or subtracted. For example, the top-left entry of A+B is 2 + (-4) = -2.

The last part of the code also demonstrates scalar multiplication, where the scalar 2 is multiplied by every single entry in matrix A. These basic operations are the building blocks for more complex matrix algebra and are frequently used in digital signal processing, for instance, to scale or offset signals.

Scalar Multiplication

If \(A\) is any matrix and \(c\) is any scalar (real number), then the product \(cA\) is the matrix obtained by multiplying each entry of \(A\) by \(c\). \(cA\) is called a scalar multiple of \(A\).

In matrix notation: \((cA)_{ij} = c(A)_{ij} = ca_{ij}\).

Example: For \(A = \left[\begin{smallmatrix} 2 & 3 & 4 \\ 1 & 3 & 1 \end{smallmatrix}\right]\), then \(2A = \left[\begin{smallmatrix} 4 & 6 & 8 \\ 2 & 6 & 2 \end{smallmatrix}\right]\).

It is common practice to denote \((-1)B\) by \(-B\).

Application in ECE: Scaling measurements from sensors, adjusting signal amplitudes, or weighting different components in a multi-input system.

Scalar multiplication is straightforward: to multiply a matrix by a scalar, you simply multiply every single entry in the matrix by that scalar. It’s like uniformly scaling every element of the matrix.

In an ECE context, imagine you have a matrix representing voltage readings from multiple points in a circuit. If you want to convert these readings from millivolts to volts, you would multiply the entire matrix by a scalar factor of \(1/1000\). Or, if you need to double the gain of an amplifier that outputs signals represented as a matrix, you’d multiply that matrix by 2. It’s a very common operation for adjusting magnitudes.

Matrix Multiplication (Product)

This is the most complex, but most powerful, basic matrix operation.

If \(A\) is an \(m \times r\) matrix and \(B\) is an \(r \times n\) matrix, then the product \(AB\) is the \(m \times n\) matrix.

Crucial Compatibility Rule: The number of columns of the first matrix (\(A\)’s columns, \(r\)) must equal the number of rows of the second matrix (\(B\)’s rows, \(r\)).

• If compatible (\(m \times \mathbf{r} \cdot \mathbf{r} \times n\)), the resulting product matrix has size \(m \times n\).

Row-Column Rule: To find the entry \((AB)_{ij}\) (row \(i\), column \(j\) of \(AB\)), you single out row \(i\) from \(A\) and column \(j\) from \(B\). Multiply corresponding entries from the selected row and column, and then add up the resulting products. \[ (AB)_{ij} = a_{i1}b_{1j} + a_{i2}b_{2j} + \dots + a_{ir}b_{rj} \]

Matrix multiplication is fundamentally different from the previous operations because it’s not element-wise. It’s a “row-column” operation and is the cornerstone of linear transformations.

The most critical rule is the compatibility constraint: the “inner dimensions” must match. If A is m x r and B is s x n, then for AB to be defined, r must equal s. If they don’t match, the multiplication is undefined.

The row-column rule is what defines the actual computation. To get any specific entry in the product matrix, say the entry in the \(i\)-th row and \(j\)-th column, you take the \(i\)-th row of the first matrix (A) and the \(j\)-th column of the second matrix (B). You then perform a dot product: multiply the first element of the row by the first element of the column, the second by the second, and so on, then sum all these products. This might seem abstract, but it’s how linear transformations are applied, how signals are convolved, and how components in a system interact.

Example 5: Multiplying Matrices

Find \(AB\) for: \[ A={\left[\begin{array}{l l l}{1}&{2}&{4}\\ {2}&{6}&{0}\end{array}\right]},\quad B={\left[\begin{array}{l l l l}{4}&{1}&{4}&{3}\\ {0}&{-1}&{3}&{1}\\ {2}&{7}&{5}&{2}\end{array}\right]} \] \(A\) is \(2 \times \mathbf{3}\) and \(B\) is \(\mathbf{3} \times 4\). Product \(AB\) will be \(2 \times 4\).

Let’s compute \((AB)_{23}\) (row 2, col 3): \((2 \cdot 4) + (6 \cdot 3) + (0 \cdot 5) = 8 + 18 + 0 = 26\)

Let’s compute \((AB)_{14}\) (row 1, col 4): \((1 \cdot 3) + (2 \cdot 1) + (4 \cdot 2) = 3 + 2 + 8 = 13\)

The full product: \[ A B={\left[\begin{array}{l l l l}{12}&{27}&{30}&{13}\\ {8}&{-4}&{26}&{12}\end{array}\right]} \]

Applications in ECE:

• Transformations: Rotating, scaling, or projecting vectors in graphics and robotics.
• Circuit Analysis: Solving network equations (e.g., \(V = IZ\)), impedance, and admittance matrices.
• Digital Signal Processing (DSP): Filtering operations, Fourier transforms can be expressed as matrix multiplications.

Let’s see this in action with an example. Matrix A is 2x3, and Matrix B is 3x4. Since the inner dimensions (3 and 3) match, the product AB is defined and will be a 2x4 matrix.

Let’s calculate a couple of entries, for instance, the entry in the second row, third column of AB. We take the second row of A: [2 6 0] and the third column of B: [4 3 5]^T. We then perform the dot product: (2*4) + (6*3) + (0*5) = 8 + 18 + 0 = 26. This is the entry (AB)23.

This seemingly complex operation is essential in ECE because it represents the application of linear transformations.

• In computer graphics, multiplying a vector (like a point in 3D space) by a transformation matrix rotates or scales that point.
• In circuit analysis, matrix multiplication is used to represent the relationship between current, voltage, and impedance in complex networks, allowing you to solve for unknown quantities using \(V=IZ\) in matrix form.
• In DSP, applying filters to signals, or performing Fourier transforms, are often conceptually (and computationally) done via matrix multiplication. It’s how signals are manipulated and transformed.

Interactive Example: Matrix Multiplication

Let’s use Python to compute the full matrix product:

Here, we use NumPy again, which provides a very convenient @ operator for matrix multiplication.

The code precisely demonstrates the calculation from the previous slide. You can see how NumPy handles all the dot products for each entry, giving us the full 2x4 product matrix. It also verifies the dimensions, showing that a (2x3) multiplied by a (3x4) yields a (2x4) matrix.

This interactivity allows you to verify the row-column rule yourself by picking any entry and manually computing its value. Understanding the inner workings of matrix multiplication is key to grasping how linear transformations map inputs to outputs in various engineering systems.

Matrix Multiplication by Columns and By Rows

Matrix multiplication can be viewed column by column or row by row:

If \(AB\) is defined,

1. \(j\)-th column of \(AB\) is \(A\) times the \(j\)-th column of \(B\). \[ AB = A[\mathbf{b}_1\quad \mathbf{b}_2\quad \dots \quad \mathbf{b}_n] = [A\mathbf{b}_1\quad A\mathbf{b}_2\quad \dots \quad A\mathbf{b}_n] \]
2. \(i\)-th row of \(AB\) is \(i\)-th row of \(A\) times \(B\). \[ AB = \left[ \begin{array}{c}\mathbf{a}_1\\ \mathbf{a}_2\\ \vdots \\ \mathbf{a}_m \end{array} \right]B = \left[ \begin{array}{c}\mathbf{a}_1B\\ \mathbf{a}_2B\\ \vdots \\ \mathbf{a}_mB \end{array} \right] \]

This is useful for computing specific rows or columns without the full product.

While the row-column rule calculates each entry, there are also useful ways to think about computing entire rows or columns of the product. These ideas are particularly important for understanding the structure of matrix transformations.

The first formula states that each column of the product \(AB\) is formed by multiplying matrix \(A\) by the corresponding column of matrix \(B\). This means if you want just the second column of \(AB\), you multiply \(A\) by the second column vector of \(B\).

Similarly, the second formula indicates that each row of \(AB\) is obtained by multiplying the corresponding row of \(A\) by the entire matrix \(B\). This is beneficial when you only need specific output rows translated by the transformation. These different perspectives are very important in optimizing computations, particularly in parallel computing environments. In signal processing, for instance, you might only care about the output of a specific filter (one row) applied to different input signals.

Matrix Products as Linear Combinations

A linear combination of matrices \(A_1, \ldots, A_r\) with scalars \(c_1, \ldots, c_r\) is \(c_1A_1 + c_2A_2 + \dots + c_rA_r\).

Theorem 1.3.1: If \(A\) is an \(m \times n\) matrix and \(\mathbf{x}\) is an \(n \times 1\) column vector, then the product \(A\mathbf{x}\) can be expressed as a linear combination of the column vectors of \(A\), in which the coefficients are the entries of \(\mathbf{x}\).

\(A\mathbf{x} = x_1(\text{col }_1 \text{ of } A) + x_2(\text{col }_2 \text{ of } A) + \dots + x_n(\text{col }_n \text{ of } A)\)

Example 8: \[ {\left[\begin{array}{l l l}{-1}&{3}&{2}\\ {1}&{2}&{-3}\\ {2}&{1}&{-2}\end{array}\right]}{\left[\begin{array}{l}{2}\\ {-1}\\ {3}\end{array}\right]}={\left[\begin{array}{l}{1}\\ {-9}\\ {-3}\end{array}\right]} \] This is equivalent to: \[ 2{\left[\begin{array}{l}{-1}\\ {1}\\ {2}\end{array}\right]} - 1{\left[\begin{array}{l}{3}\\ {2}\\ {1}\end{array}\right]} + 3{\left[\begin{array}{l}{2}\\ {-3}\\ {-2}\end{array}\right]} = {\left[\begin{array}{l}{1}\\ {-9}\\ {-3}\end{array}\right]} \]

Application in ECE: This is fundamental for understanding superposition principles in circuits, representing signals as combinations of basis functions (like Fourier series), or combining effects of multiple inputs in control systems.

This concept is incredibly important and is often revisited throughout linear algebra: matrix products can be seen as linear combinations. A linear combination, in simple terms, is a sum of vectors or matrices scaled by constants.

Theorem 1.3.1 is key: If you multiply a matrix \(A\) by a column vector \(\mathbf{x}\), the result (\(A\mathbf{x}\)) is a linear combination of the columns of \(A\), where the scaling coefficients are the entries of \(\mathbf{x}\). This provides a powerful geometric and conceptual understanding of what matrix multiplication does.

The example clearly illustrates this. The product of the matrix and vector is exactly equal to scaling each column of the matrix by the corresponding entry in the vector and then summing those scaled columns. This is not just a mathematical curiosity; it has profound implications: * In circuit analysis, if the columns of A represent responses to individual input signals, then \(A\mathbf{x}\) represents the total response to a combined input signal \(\mathbf{x}\), illustrating the superposition principle. * In signal processing, it allows us to express complex signals as combinations of simpler “basis” signals, which is the core idea behind Fourier analysis and wavelets. * In control systems, it helps analyze how different control inputs combine to affect the system’s state.

Matrix Form of a Linear System

Any system of \(m\) linear equations in \(n\) unknowns can be written compactly as a single matrix equation: \[ A\mathbf{x} = \mathbf{b} \] Where:

• \(A\) is the coefficient matrix (\(m \times n\)).
• \(\mathbf{x}\) is the unknown vector (\(n \times 1\)).
• \(\mathbf{b}\) is the constant vector (\(m \times 1\)).

Example: \[ \begin{array}{c}x_{1} - 2x_{2} + x_{3} = 0\\ 2x_{1} + 0x_{2} + 3x_{3} = 5\end{array} \quad \rightarrow \quad \left[ \begin{array}{r r r}{1} & -2 & 1\\ 2 & 0 & 3 \end{array} \right]\left[ \begin{array}{c}{x_{1}}\\{x_{2}}\\{x_{3}}\end{array} \right] = \left[ \begin{array}{c}{0}\\{5}\end{array} \right] \]

The augmented matrix for the system is \([A \mid \mathbf{b}]\).

Application in ECE: This compact notation simplifies the representation and analysis of large, complex systems found in circuits, control, and power systems. Rather than writing out pages of individual equations, we can deal with the entire system as one matrix equation.

Bringing it back to our first topic, systems of linear equations. Matrix multiplication provides an elegant way to represent an entire system of linear equations as a single matrix equation: \(A\mathbf{x} = \mathbf{b}\).

Here, \(A\) collects all the coefficients, \(\mathbf{x}\) is the column vector of all your unknown variables, and \(\mathbf{b}\) is the column vector of all the constants on the right-hand side. The example vividly shows how a typical system translates into this matrix form.

This matrix representation is more than just a compact notation; it’s a fundamental shift in how we think about solving these problems. Instead of seeing individual equations, we see a single transformation (\(A\)) acting on an input (\(\mathbf{x}\)) to produce an output (\(\mathbf{b}\)). This unified view is crucial in all areas of ECE, from power system load flow analysis to solving for node voltages in a circuit network, where you might be dealing with systems of hundreds or thousands of equations.

Transpose of a Matrix

The transpose of an \(m \times n\) matrix \(A\), denoted by \(A^T\), is the \(n \times m\) matrix obtained by interchanging the rows and columns of \(A\).

• The first column of \(A^T\) is the first row of \(A\), and so on.
• The entry in row \(i\), column \(j\) of \(A^T\) is \((A^T)_{ij} = (A)_{ji}\).

Example: \[ A=\left[\begin{array}{c c c}{{1}}&{{-2}}&{{4}}\\ {{3}}&{{7}}&{{0}}\\ {{-5}}&{{8}}&{{6}}\end{array}\right] \quad \rightarrow \quad A^{T}=\left[\begin{array}{c c c}{{1}}&{{3}}&{{-5}}\\ {{-2}}&{{7}}&{{8}}\\ {{4}}&{{0}}&{{6}}\end{array}\right] \] For a square matrix, this is like “reflecting” the matrix entries about its main diagonal.

Application in ECE:

• Signal Processing: Analyzing properties of signals and systems, e.g., in covariance matrices for noisy signals.
• Data Science/ML: Reshaping data for algorithms, in least squares regression and principal component analysis (PCA).
• Quantum Mechanics: Representing operators and states in quantum computing.

Transpose of a Matrix

The transpose is an operation unique to matrices and has no direct scalar counterpart. It simply flips the matrix over its main diagonal, effectively swapping rows and columns. What was the first row becomes the first column, and so on.

The example shows how a square matrix is reflected. For a non-square matrix, the dimensions also swap: a 3x2 matrix becomes a 2x3 matrix when transposed.

In ECE, the transpose is extremely common:

• In signal processing, when you’re dealing with matrices of data, you often need to transpose them to align dimensions for operations like matrix multiplication, particularly for calculating correlation or covariance matrices. For example, if you have a matrix where rows are time points and columns are sensor data, transposing it means columns are time points and rows are sensor data.
• It’s integral to the mathematics of least squares regression, where you try to find the best fit line for a set of data points, common in system identification and control.
• In quantum computing, the adjoint (conjugate transpose) of a matrix is a fundamental operation representing a reverse process or observation.

Trace of a Matrix

If \(A\) is a square matrix, then the trace of \(A\), denoted by \(\operatorname{tr}(A)\), is defined to be the sum of the entries on the main diagonal of \(A\). The trace of \(A\) is undefined if \(A\) is not a square matrix.

Example: For \(A = \left[\begin{smallmatrix} a_{11} & a_{12} & a_{13} \\ a_{21} & a_{22} & a_{23} \\ a_{31} & a_{32} & a_{33} \end{smallmatrix}\right]\), \(\operatorname{tr}(A) = a_{11} + a_{22} + a_{33}\).

For \(B = \left[\begin{smallmatrix} -1 & 2 & 7 & 0 \\ 3 & 5 & -8 & 4 \\ 1 & 2 & 7 & -3 \\ 4 & -2 & 1 & 0 \end{smallmatrix}\right]\), \(\operatorname{tr}(B) = -1 + 5 + 7 + 0 = 11\).

Application in ECE:

• Control Systems: Used in system analysis, e.g., in stability criteria and Lyapunov equations.
• Quantum Mechanics: Represents observable quantities; used in density matrices.
• Numerical Analysis: Useful properties for eigenvalue calculations.

Trace of a Matrix

The trace is another special operation, defined only for square matrices. It’s simply the sum of the elements along the main diagonal.

The example clearly shows how to compute the trace by summing the highlighted diagonal elements. For a larger matrix, it’s the sum of \(a_{11}, a_{22}, a_{33}\), and so on.

While seemingly simple, the trace has deep theoretical significance in various engineering fields: * In control theory, the trace of certain matrices can reveal information about system stability or how quickly a system converges to a desired state. * In quantum mechanics, the trace of a density matrix gives the total probability of all states, which must be 1. * It plays a role in finding eigenvalues, which are critical for understanding the fundamental behavior of linear systems like resonators or RLC circuits. Python’s NumPy library has a convenient np.trace() function that computes this directly.

Conclusion

• Matrices are powerful data structures: Representing data and relationships in a structured way (e.g., sensor arrays, system states).
• Core Operations:
  • Addition & Subtraction: Element-wise (same size required).
  • Scalar Multiplication: Scales all entries by a constant.
  • Matrix Multiplication: Row-column rule (inner dimensions must match)  —  represents linear transformations and data interaction.
• Conceptual Depth: Matrix products as linear combinations (superposition, basis vectors).
• Special Operations:
  • Transpose (\(A^T\)): Swaps rows and columns (data reorientation).
  • Trace (\(\operatorname{tr}(A)\)): Sum of main diagonal elements (for square matrices, revealing system properties).
• Ubiquitous in ECE: These operations form the bedrock of numerical methods for solving problems in circuits, signals, control, robotics, imaging, communication, and machine learning.

Mastering these matrix operations is essential for your engineering toolkit!

To wrap up our discussion on matrices and matrix operations from Section 1.3:

We started by defining matrices as flexible data structures for organizing numbers, crucial for representing everything from sensor data to system coefficients in ECE.

We covered the fundamental arithmetic operations: * Addition and subtraction are element-wise, meaning they combine corresponding entries. These are like combining multiple sets of measurements or changes in system states. * Scalar multiplication uniformly scales data, * And most importantly, matrix multiplication, which is a row-column operation with strict dimension compatibility rules. This operation is the mathematical language of linear transformations, describing how systems evolve or how signals are processed.

We also discussed the conceptual power of viewing matrix products as linear combinations of column vectors, which directly translates to the principle of superposition in many physical systems.

Finally, we introduced the transpose, which reorients matrices, and the trace, which sums diagonal elements, both having significant applications in advanced analysis across ECE.

These operations are not just abstract mathematical concepts; they are the fundamental “verbs” of linear algebra that you will use daily in circuit simulation, signal processing, control system design, image processing, and even in novel areas like quantum computing and AI. Without a solid grasp of these operations, tackling complex engineering challenges becomes significantly harder. Keep practicing these, as they are foundational for every topic moving forward.

Thank you! I look forward to seeing you next time as we delve into properties of matrix operations.

Exercises

grasple_exercise_3_2_2

To compute \(c_1A + c_2B\).

Exercises

grasple_exercise_3_2_3

To compute \(c_1A + c_2B\).