Linear Algebra

1.8 Matrix Transformations

Imron Rosyadi

Linear Algebra in ECE

1.8 Matrix Transformations

Imron Rosyadi

Understanding how matrices elegantly represent geometric and algebraic transformations, crucial for engineering applications.

Good morning everyone! Today, we’re delving into a concept that truly bridges the gap between abstract linear algebra and its concrete applications in Electrical and Computer Engineering: matrix transformations. These are special functions arising from matrix multiplication, and they are absolutely fundamental to understanding everything from image processing and computer graphics to robotics, control systems, and machine learning. We’ll see how matrix transformations provide a powerful and intuitive way to describe linear mappings between vector spaces.

Introduction: Vectors in \(R^n\)

• An ordered \(n\)-tuple of real numbers can be seen as a vector.
  • Example: \((s_1, s_2, \ldots, s_n)\)
• The set of all ordered \(n\)-tuples forms the space \(R^n\).
• We primarily use column vector form: \[ \mathbf{x} = \left[ \begin{array}{c}s_{1} \\ s_{2} \\ \vdots \\ s_{n} \end{array} \right] \]

Introduction: Vectors in \(R^n\)

• Standard Basis Vectors for \(R^n\): \(\mathbf{e}_1, \mathbf{e}_2, \ldots, \mathbf{e}_n\).
  • \(\mathbf{e}_i\) has a 1 in the \(i\)-th position and zeros elsewhere.
  • Any vector \(\mathbf{x}\) in \(R^n\) can be uniquely written as: \[ \mathbf{x} = x_{1} \mathbf{e}_{1} + x_{2} \mathbf{e}_{2} + \dots + x_{n} \mathbf{e}_{n} \] For example, in \(R^3\): \(\mathbf{e}_1 = \left[ \begin{smallmatrix} 1 \\ 0 \\ 0 \end{smallmatrix} \right]\), \(\mathbf{e}_2 = \left[ \begin{smallmatrix} 0 \\ 1 \\ 0 \end{smallmatrix} \right]\), \(\mathbf{e}_3 = \left[ \begin{smallmatrix} 0 \\ 0 \\ 1 \end{smallmatrix} \right]\).

Before we talk about transformations, let’s quickly recall our concept of vectors. In linear algebra, a vector in \(R^n\) is simply an ordered list of \(n\) real numbers. While we can write them as tuples, for matrix operations, we almost always prefer the column vector form. These vectors live in a space called \(R^n\). For example, \(R^2\) is the Cartesian plane, and \(R^3\) is 3D space.

A crucial concept is the ‘standard basis vectors’ for \(R^n\). These are like the fundamental building blocks of the space. For example, in \(R^3\), we have \(\mathbf{e}_1\), \(\mathbf{e}_2\), and \(\mathbf{e}_3\). Any vector in \(R^n\) can be expressed as a unique linear combination of these basis vectors. This property will be incredibly important when we learn how to define matrix transformations.

Functions and Transformations

Recall that a function is a rule that associates each element of a set A (the domain) with exactly one element in a set B (the codomain). The set of all possible output elements is called the range.

If a function \(f\) maps from \(R^n\) to \(R^m\), we call \(f\) a transformation from \(R^n\) to \(R^m\), denoted \(f: R^n \to R^m\). If \(m=n\), it’s sometimes called an operator on \(R^n\).

We’re all familiar with functions like \(y = f(x)\). In linear algebra, we’re interested in functions where the input is a vector and the output is also a vector. When the domain is \(R^n\) and the codomain is \(R^m\), we call such a function a ‘transformation’. This terminology suggests a change, a mapping from one space to another. If the input and output spaces are the same, \(R^n\) to \(R^n\), we often refer to it as an ‘operator’. Think of it as taking an input vector, applying a rule, and getting an output vector.

Matrix Transformations: The Core Idea

A matrix transformation is a transformation from \(R^n\) to \(R^m\) that arises from matrix multiplication.

Consider a system of linear equations: \[ \begin{array}{l}w_{1} = a_{11}x_{1} + a_{12}x_{2} + \dots + a_{1n}x_{n} \\ w_{2} = a_{21}x_{1} + a_{22}x_{2} + \dots + a_{2n}x_{n} \\ \vdots \qquad \vdots \qquad \vdots \qquad \vdots \\ w_{m} = a_{m1}x_{1} + a_{m2}x_{2} + \dots + a_{mn}x_{n} \end{array} \] This can be written in matrix notation as \(\mathbf{w} = A\mathbf{x}\), where: \[ \left[ \begin{array}{c}w_{1} \\ w_{2} \\ \vdots \\ w_{m} \end{array} \right] = \left[ \begin{array}{cccc}a_{11} & a_{12} & \dots & a_{1n} \\ a_{21} & a_{22} & \dots & a_{2n} \\ \vdots & \vdots & & \vdots \\ a_{m1} & a_{m2} & \dots & a_{mn} \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \\ \vdots \\ x_{n} \end{array} \right] \]

Matrix Transformations: The Core Idea

We view this as a transformation \(T_A: R^n \to R^m\) such that \(T_A(\mathbf{x}) = A\mathbf{x}\).

The core idea is that any system of linear equations can be expressed as a matrix-vector product \(A\mathbf{x}\). When we conceive of this product as an input-output mapping, where \(\mathbf{x}\) is the input vector from \(R^n\) and \(\mathbf{w}\) is the output vector from \(R^m\), we have a matrix transformation. The matrix \(A\) ‘transforms’ vector \(\mathbf{x}\) into vector \(\mathbf{w}\). The diagram visually shows this mapping process.

Example 1: A Matrix Transformation from \(R^4\) to \(R^3\)

Consider the transformation defined by: \[ \begin{array}{l}w_{1} = 2x_{1} - 3x_{2} + x_{3} - 5x_{4} \\ w_{2} = 4x_{1} + x_{2} - 2x_{3} + x_{4} \\ w_{3} = 5x_{1} - x_{2} + 4x_{3} \end{array} \]

Example 1: A Matrix Transformation from \(R^4\) to \(R^3\)

This transformation maps vectors from \(R^4\) to \(R^3\). Its matrix form is \(\mathbf{w} = A\mathbf{x}\): \[ \left[ \begin{array}{c}w_{1} \\ w_{2} \\ w_{3} \end{array} \right] = \left[ \begin{array}{cccc}2 & -3 & 1 & -5 \\ 4 & 1 & -2 & 1 \\ 5 & -1 & 4 & 0 \end{array} \right]\left[ \begin{array}{c}x_{1} \\ x_{2} \\ x_{3} \\ x_{4} \end{array} \right] \] So, the transformation matrix is \(A = \left[ \begin{array}{cccc}2 & -3 & 1 & -5 \\ 4 & 1 & -2 & 1 \\ 5 & -1 & 4 & 0 \end{array} \right]\).

Let’s find the image of \(\mathbf{x} = \left[ \begin{smallmatrix} 1 \\ -3 \\ 0 \\ 2 \end{smallmatrix} \right]\) under \(T_A\).

Here’s a concrete example. We have a set of equations defining \(w_1, w_2, w_3\) in terms of \(x_1, x_2, x_3, x_4\). This forms a transformation from \(R^4\) to \(R^3\). We can easily extract the coefficient matrix \(A\). Now, to find the image of a specific vector \(\mathbf{x}\), we just perform the matrix-vector multiplication \(A\mathbf{x}\).

Example 1: Python Calculation

Compute \(T_A(\mathbf{x}) = A\mathbf{x}\) for the given \(A\) and \(\mathbf{x}\).

The image of \(\mathbf{x}\) is \(\mathbf{w} = \left[ \begin{smallmatrix} 1 \\ 3 \\ 8 \end{smallmatrix} \right]\).

Using numpy, this is a straightforward calculation. We define the transformation matrix \(A\) and the input vector \(\mathbf{x}\). The @ operator in Python performs matrix multiplication. The result, \(\mathbf{w}\), is the output vector. This direct computation is what happens behind the scenes in many ECE applications when transforming data, for instance, applying a filter to a signal represented as a vector, or transforming coordinates in robotics.

Special Matrix Transformations

• Zero Transformation (\(T_0: R^n \to R^m\)): If \(0\) is the \(m \times n\) zero matrix, then \(T_0(\mathbf{x}) = 0\mathbf{x} = \mathbf{0}\). (Maps every vector in \(R^n\) to the zero vector in \(R^m\).)
• Identity Operator (\(T_I: R^n \to R^n\)): If \(I\) is the \(n \times n\) identity matrix, then \(T_I(\mathbf{x}) = I\mathbf{x} = \mathbf{x}\). (Maps every vector in \(R^n\) to itself.)

Just like with numbers, we have special matrices that represent ‘nothing’ or ‘identity’. The zero matrix, when used as a transformation, maps any input vector to the zero vector. This can represent, for example, a system with no output regardless of input. The identity matrix, as a transformation, simply maps a vector to itself, meaning no change occurs. This is like a pass-through in a signal chain.

Properties of Matrix Transformations (Theorem 1.8.1)

For every matrix \(A\), the matrix transformation \(T_A: R^n \to R^m\) has the following properties:

1. \(T_A(\mathbf{0}) = \mathbf{0}\)
2. \(T_A(k\mathbf{u}) = kT_A(\mathbf{u})\) (Homogeneity property)
3. \(T_A(\mathbf{u} + \mathbf{v}) = T_A(\mathbf{u}) + T_A(\mathbf{v})\) (Additivity property)
4. \(T_A(\mathbf{u} - \mathbf{v}) = T_A(\mathbf{u}) - T_A(\mathbf{v})\)

These properties are direct consequences of matrix arithmetic axioms. Collectively, properties (b) and (c) are known as linearity conditions. They imply that matrix transformations preserve linear combinations: \[ T_A(k_1\mathbf{u}_1 + \dots + k_r\mathbf{u}_r) = k_1T_A(\mathbf{u}_1) + \dots + k_rT_A(\mathbf{u}_r) \]

These properties are fundamental, stemming directly from how matrix multiplication works. The homogeneity property means scaling the input vector by a scalar \(k\) before applying the transformation is the same as scaling the output vector by \(k\) after the transformation. The additivity property means transforming the sum of two vectors is the same as summing their individual transformations. These two properties, homogeneity and additivity, define what we call a ‘linear transformation’. They allow us to break down complex inputs into simpler components, transform them, and then combine the results, which is key in superposition, a common concept in circuit analysis.

Linear Transformations Defined

Not all transformations are matrix transformations. For example, \(w_1 = x_1^2 + x_2^2\) is not. This leads to two crucial questions:

1. How do we know if a transformation \(T: R^n \to R^m\) is a matrix transformation?
2. If it is, how do we find its unique matrix?

THEOREM 1.8.2: A transformation \(T: R^n \to R^m\) is a matrix transformation if and only if it satisfies the following linearity conditions:
(i) \(T(\mathbf{u} + \mathbf{v}) = T(\mathbf{u}) + T(\mathbf{v})\) (Additivity)
(ii) \(T(k\mathbf{u}) = k T(\mathbf{u})\) (Homogeneity)

THEOREM 1.8.3:
Every linear transformation from \(R^n\) to \(R^m\) is a matrix transformation, and conversely, every matrix transformation from \(R^n\) to \(R^m\) is a linear transformation.

This theorem is the bridge. It states that matrix transformations are exactly the same as linear transformations. The two conditions, additivity and homogeneity, are the litmus test. If a transformation satisfies these two seemingly simple rules, then it’s a linear transformation, and it must have a corresponding matrix. This is incredibly powerful. No matter how complicated the linear transformation seems, as long as it respects scaling and addition, we can represent it with a matrix.

Geometric Interpretation (Mapping Points)

A matrix transformation \(T_A: R^n \to R^m\) maps each vector (point) in \(R^n\) into a vector (point) in \(R^m\). This is the basis of transformations in computer graphics and robotics.

From a geometric perspective, a matrix transformation takes a point (or vector) in \(R^n\) and maps it to a new point (or vector) in \(R^m\). Think of rotating an object in 3D space, scaling an image, or translating a robot arm. These are all examples of geometric transformations that can be represented by matrices. This mapping concept is central to understanding how coordinates change under various operations.

Uniqueness of the Standard Matrix (Theorem 1.8.4)

If \(T_A: R^n \to R^m\) and \(T_B: R^n \to R^m\) are matrix transformations, and if \(T_A(\mathbf{x}) = T_B(\mathbf{x})\) for every vector \(\mathbf{x}\) in \(R^n\), then \(A = B\).

Significance: There is a one-to-one correspondence between linear transformations from \(R^n\) to \(R^m\) and \(m \times n\) matrices. Every such linear transformation arises from exactly one \(m \times n\) matrix. This matrix is known as the standard matrix for the transformation.

This theorem is incredibly important because it tells us that the matrix representing a linear transformation is unique. If two linear transformations map every vector to the same image, then their representing matrices must be identical. This means we don’t have to worry about finding multiple matrices for the same transformation. It establishes a perfect one-to-one link between all linear transformations and all matrices of the appropriate size. This uniquely defined matrix is called the standard matrix.

Procedure for Finding Standard Matrices

To find the standard matrix \(A\) for a linear transformation \(T: R^n \to R^m\):

Step 1: Find the images of the standard basis vectors for \(R^n\): \(T(\mathbf{e}_1), T(\mathbf{e}_2), \ldots, T(\mathbf{e}_n)\).

Recall $\mathbf{e}_1 = \left[\begin{smallmatrix} 1 \\ 0 \\ \vdots \\ 0 \end{smallmatrix}\right], \ldots, \mathbf{e}_n = \left[\begin{smallmatrix} 0 \\ \vdots \\ 0 \\ 1 \end{smallmatrix}\right]$. These vectors are from $R^n$. The images $T(\mathbf{e}_i)$ will be vectors in $R^m$.

Step 2: Construct the matrix \(A\) that has these images as its successive columns: \[ A = [T(\mathbf{e}_1) \mid T(\mathbf{e}_2) \mid \dots \mid T(\mathbf{e}_n)] \] This matrix \(A\) is the standard matrix for \(T\).

This is the practical algorithm for finding the standard matrix. The key insight is that because linear transformations preserve linear combinations, knowing how the transformation acts on the simple basis vectors \(\mathbf{e}_i\) tells you everything you need to know about how it acts on any vector. You simply apply the transformation rule to each basis vector, and the resulting image vectors become the columns of your standard matrix. This procedure is widely used in ECE to derive transformation matrices for various operations.

Example 4: Finding a Standard Matrix

Find the standard matrix \(A\) for the linear transformation \(T: R^2 \to R^3\) defined by: \[ T\left(\left[ \begin{array}{c}{x_{1}}\\ {x_{2}} \end{array} \right]\right) = \left[ \begin{array}{c}{2x_{1} + x_{2}}\\ {x_{1} - 3x_{2}}\\ {-x_{1} + x_{2}} \end{array} \right] \]

Example 4: Finding a Standard Matrix

Step 1: Find images of standard basis vectors for \(R^2\). \(\mathbf{e}_1 = \left[ \begin{smallmatrix} 1 \\ 0 \end{smallmatrix} \right]\) and \(\mathbf{e}_2 = \left[ \begin{smallmatrix} 0 \\ 1 \end{smallmatrix} \right]\).

\(T(\mathbf{e}_1) = T\left(\left[ \begin{smallmatrix} 1 \\ 0 \end{smallmatrix} \right]\right) = \left[ \begin{array}{c}{2(1) + 0}\\ {1 - 3(0)}\\ {-1 + 0} \end{array} \right] = \left[ \begin{array}{r}{2}\\ {1}\\ {-1} \end{array} \right]\) \(T(\mathbf{e}_2) = T\left(\left[ \begin{smallmatrix} 0 \\ 1 \end{smallmatrix} \right]\right) = \left[ \begin{array}{c}{2(0) + 1}\\ {0 - 3(1)}\\ {-0 + 1} \end{array} \right] = \left[ \begin{array}{r}{1}\\ {-3}\\ {1} \end{array} \right]\)

Step 2: Construct the standard matrix A. \(A = [T(\mathbf{e}_1) \mid T(\mathbf{e}_2)]\) \[ A = \left[ \begin{array}{rr}{2} & {1}\\ {1} & {-3}\\ {-1} & {1} \end{array} \right] \]

Let’s apply the procedure. Our transformation takes a 2-D vector and outputs a 3-D vector. So our standard matrix A will be a 3x2 matrix. We need to find how T acts on the standard basis vectors of \(R^2\). First, for \(\mathbf{e}_1\), we substitute \(x_1=1, x_2=0\) into the definition of \(T\). This gives us the first column of \(A\). Then, for \(\mathbf{e}_2\), we substitute \(x_1=0, x_2=1\). This gives us the second column of \(A\). The resulting matrix \(A\) is the unique standard matrix for this transformation.

Example 5: Computing with Standard Matrices

For the linear transformation in Example 4, use the standard matrix \(A\) to find \(T\left(\left[ \begin{smallmatrix} 1 \\ 4 \end{smallmatrix} \right]\right)\).

The standard matrix is \(A = \left[ \begin{array}{rr}{2} & {1}\\ {1} & {-3}\\ {-1} & {1} \end{array} \right]\). Let \(\mathbf{x} = \left[ \begin{smallmatrix} 1 \\ 4 \end{smallmatrix} \right]\).

The transformation is multiplication by \(A\), so \(T(\mathbf{x}) = A\mathbf{x}\).

The image is \(T\left(\left[ \begin{smallmatrix} 1 \\ 4 \end{smallmatrix} \right]\right) = \left[ \begin{smallmatrix} 6 \\ -11 \\ 3 \end{smallmatrix} \right]\).

Now that we have the standard matrix \(A\), applying the transformation to any vector \(\mathbf{x}\) is simply a matrix-vector multiplication \(A\mathbf{x}\). This is computationally very efficient, especially for many different input vectors. Our numpy calculation confirms the result. This approach is what enables efficient transformations in applications like graphics rendering, where an image (a collection of points/vectors) is transformed by a single matrix.

Example 6: Finding a Standard Matrix (Comma-Delimited Form)

Rewrite the transformation \(T(x_1, x_2) = (3x_1 + x_2, 2x_1 - 4x_2)\) in column-vector form and find its standard matrix.

Step 1: Rewrite in column-vector form. \[ T\left(\left[ \begin{array}{c}{x_{1}}\\ {x_{2}} \end{array} \right]\right) = \left[ \begin{array}{c}{3x_{1} + x_{2}}\\ {2x_{1} - 4x_{2}} \end{array} \right] \]

Example 6: Finding a Standard Matrix (Comma-Delimited Form)

Step 2: Identify the coefficients to form the matrix directly. We observe that the output vector is a linear combination of \(x_1\) and \(x_2\) with coefficients forming columns: \[ \left[ \begin{array}{c}{3x_{1} + x_{2}}\\ {2x_{1} - 4x_{2}} \end{array} \right] = x_1\left[ \begin{array}{c}{3}\\ {2} \end{array} \right] + x_2\left[ \begin{array}{r}{1}\\ {-4} \end{array} \right] = \left[ \begin{array}{rr}{3} & {1}\\ {2} & {-4} \end{array} \right]\left[ \begin{array}{c}{x_{1}}\\ {x_{2}} \end{array} \right] \] Thus, the standard matrix is: \[ A = \left[ \begin{array}{rr}{3} & {1}\\ {2} & {-4} \end{array} \right] \]

Alternatively, explicitly apply the procedure:
\(T(\mathbf{e}_1) = T(1,0) = (3(1)+0, 2(1)-4(0)) = (3,2)\), so \(\left[ \begin{smallmatrix} 3 \\ 2 \end{smallmatrix} \right]\).
\(T(\mathbf{e}_2) = T(0,1) = (3(0)+1, 2(0)-4(1)) = (1,-4)\), so \(\left[ \begin{smallmatrix} 1 \\ -4 \end{smallmatrix} \right]\).
The standard matrix is then \(\left[ \begin{array}{rr}{3} & {1}\\ {2} & {-4} \end{array} \right]\).

Sometimes transformations are given in a more compact, comma-delimited form. The first step is always to convert it to the column-vector form, which makes the coefficients more obvious. Once in that form, you can either directly write down the coefficient matrix as the standard matrix, or, if less obvious, explicitly apply the \(T(\mathbf{e}_i)\) procedure as before. The result is always the same unique standard matrix.

ECE Applications & Summary

Key Concepts for ECE:

• Computer Graphics & Image Processing: Rotations, scaling, translations (affine transformations, built on linear transformations) of 2D/3D objects and images.
• Robotics: Kinematics (mapping joint angles to end-effector positions), robot path planning, sensor data transformation.
• Control Systems: State-space representation, system linearization, transformations between different coordinate frames (e.g., in aerospace engineering).
• Signal Processing: Filtering operations (e.g., convolution), Fourier transforms, wavelet transforms can often be represented as linear transformations.
• Circuit Analysis: Impedance transformations, network analysis can involve linear mappings.
• Data Science/Machine Learning: Feature transformations, dimensionality reduction (e.g., PCA), neural network layers often perform linear transformations.

ECE Applications & Summary

Today We Covered:

• Vectors in \(R^n\) and transformations from \(R^n\) to \(R^m\).
• The definition of a matrix transformation as \(T_A(\mathbf{x}) = A\mathbf{x}\).
• The essential linearity conditions (additivity & homogeneity) that uniquely define linear transformations.
• The one-to-one correspondence between linear transformations and their standard matrices.
• A practical procedure for finding the standard matrix by transforming standard basis vectors.

Matrix transformations are everywhere in ECE. They are the mathematical backbone behind how we manipulate data, control physical systems, and render virtual environments. Whether you’re rotating a CAD model, filtering noise from a signal, designing a controller for an autonomous vehicle, or processing images, you’re constantly working with matrix transformations. Understanding them is not just an academic exercise; it’s a fundamental skill for any engineer. Today, we laid the groundwork by defining what matrix transformations are, identifying their core properties, and learning the crucial method to find their unique standard matrix.