Linear Algebra

3.4 The Geometry of Linear Systems

Imron Rosyadi

The Geometry of Linear Systems

Unveiling Vector & Parametric Equations

Introduction: Visualizing Linear Algebra

In ECE, understanding linear systems isn’t just about solving equations; it’s about visualizing their geometric interpretations.

This lecture explores how to represent solution sets of linear systems as geometric objects like points, lines, and planes.

Welcome to this session on the geometry of linear systems. For ECE students, abstract mathematical concepts become much more tangible and useful when we can visualize them. Today, we’ll bridge the gap between algebraic equations and their geometric counterparts, which is crucial for fields like signal processing, control theory, and computer graphics.

Lines in \(R^2\) and \(R^3\): Vector Form

A unique line L is determined by a point \(\mathbf{x}_0\) on the line and a non-zero vector \(\mathbf{v}\) parallel to the line.

The vector \(\mathbf{x} - \mathbf{x}_0\) is a scalar multiple of \(\mathbf{v}\). \[ \mathbf{x} - \mathbf{x}_0 = t\mathbf{v} \] This leads to the vector equation of a line: \[ \mathbf{x} = \mathbf{x}_0 + t\mathbf{v} \quad \text{(1)} \] Where \(t\) is the parameter, varying from \(-\infty\) to \(\infty\).

If \(\mathbf{x}_0 = \mathbf{0}\), the line passes through the origin: \[ \mathbf{x} = t\mathbf{v} \quad \text{(2)} \]

Figure 3.4.2

Note

To define a line in either 2D (ℝ²) or 3D (ℝ³), you need:

• A point 𝑥0 on the line.
• A direction vector 𝑣 that is parallel to the line.

Any point 𝑥 on the line satisfies: 𝑥−𝑥0=𝑡𝑣

Which rearranges to the vector equation of a line: 𝑥 =𝑥0+𝑡𝑣

Imagine a point x0 in space. Now, imagine a direction vector v. Any point x on the line starting from x0 and moving in direction v can be reached by adding a scaled version of v to x0. The scalar t dictates how far along v we move, and its variation covers the entire line. This concept is fundamental in robotics for path planning or in control systems for describing system trajectories.

Interactive Line Plot in \(R^2\)

Adjust the starting point \(\mathbf{x}_0\) and direction vector \(\mathbf{v}\) to see how the line changes.

Tip

Try changing the values of x0_val, y0_val, vx_val, and vy_val below. Observe how the line shifts and rotates.

This interactive plot allows you to directly manipulate the parameters x0 and v. Think about how changing the components of v affects the slope and direction of the line. Changing x0 simply translates the entire line without altering its orientation. This is a perfect example of how vector addition works geometrically. In signal processing, a line can represent a signal’s trajectory over time, where x0 is the initial state and v is the rate of change.

Planes in \(R^3\): Vector Form

A unique plane W is determined by a point \(\mathbf{x}_0\) in the plane and two non-collinear (not lie on the same line) vectors \(\mathbf{v}_1, \mathbf{v}_2\) parallel to the plane.

The vector \(\mathbf{x} - \mathbf{x}_0\) can be expressed as a linear combination of \(\mathbf{v}_1\) and \(\mathbf{v}_2\). \[ \mathbf{x} - \mathbf{x}_0 = t_1\mathbf{v}_1 + t_2\mathbf{v}_2 \] This leads to the vector equation of a plane: \[ \mathbf{x} = \mathbf{x}_0 + t_1\mathbf{v}_1 + t_2\mathbf{v}_2 \quad \text{(3)} \] Where \(t_1, t_2\) are parameters, varying from \(-\infty\) to \(\infty\).

If \(\mathbf{x}_0 = \mathbf{0}\), the plane passes through the origin: \[ \mathbf{x} = t_1\mathbf{v}_1 + t_2\mathbf{v}_2 \quad \text{(4)} \]

Figure 3.4.3

For a plane, instead of one direction, we need two independent directions. These vectors, v1 and v2, span the plane when added to x0. The parameters t1 and t2 allow us to reach any point within that plane. Think of this as defining a 2D surface within a 3D space. This is relevant for defining surfaces in CAD software, or understanding the solution space of certain optimization problems in ECE.

Interactive Plane Plot in \(R^3\)

Visualize a plane defined by a point and two direction vectors.

Tip

Modify the components of x0, v1, and v2 to explore different planes. Notice how v1 and v2 must be non-collinear to define a plane.

This interactive plot lets us visualize a plane in 3D space. Notice how the two vectors v1 and v2 define the orientation of the plane. If v1 and v2 were collinear, they would only define a line, not a plane. This is an important condition: the vectors must be linearly independent. This kind of visualization is critical for understanding concepts like subspaces and basis vectors in higher dimensions.

Generalizing to \(R^n\)

The concepts of lines and planes extend naturally to higher dimensions.

Definition 1: Line in \(R^n\)

If \(\mathbf{x}_0\) and \(\mathbf{v}\) are vectors in \(R^n\), and if \(\mathbf{v}\) is nonzero, then the equation \[ \mathbf{x} = \mathbf{x}_0 + t\mathbf{v} \quad \text{(5)} \] defines the line through \(\mathbf{x}_0\) that is parallel to \(\mathbf{v}\).

In the special case where \(\mathbf{x}_0 = \mathbf{0}\), the line is said to pass through the origin.

Definition 2: Plane in \(R^n\)

If \(\mathbf{x}_0\), \(\mathbf{v}_1\), and \(\mathbf{v}_2\) are vectors in \(R^n\), and if \(\mathbf{v}_1\) and \(\mathbf{v}_2\) are not collinear, then the equation \[ \mathbf{x} = \mathbf{x}_0 + t_1\mathbf{v}_1 + t_2\mathbf{v}_2 \quad \text{(6)} \] defines the plane through \(\mathbf{x}_0\) that is parallel to \(\mathbf{v}_1\) and \(\mathbf{v}_2\).

In the special case where \(\mathbf{x}_0 = \mathbf{0}\), the plane is said to pass through the origin.

While we can’t visualize \(R^n\) directly for \(n > 3\), the algebraic forms remain the same. These vector forms are powerful because they generalize easily. In ECE, working with high-dimensional data is common, for example, in machine learning or large-scale control systems, where x could represent a state vector with many components.

Example 1: Lines in \(R^2\) and \(R^3\)

Part (a): Line in \(R^2\) through origin, parallel to \(\mathbf{v} = (-2,3)\)

Vector equation: \((x,y) = t(-2,3)\)

Parametric equations: \(x = -2t, \quad y = 3t\)

Part (b): Line in \(R^3\) through \(P_0(1,2,-3)\), parallel to \(\mathbf{v} = (4,-5,1)\)

Vector equation: \((x, y, z) = (1, 2, -3) + t(4, -5, 1)\)

Parametric equations: \(x = 1 + 4t, \quad y = 2 - 5t, \quad z = -3 + t\)

Note

The parameter \(t\) allows us to trace any point on the line. Different values of \(t\) correspond to different points.

These examples illustrate how to translate between vector and parametric forms. The parametric form is particularly useful when you need to substitute coordinates into other equations or perform calculations component-wise. For instance, in trajectory planning for a drone, you might need to know its exact coordinates \((x, y, z)\) at a given time \(t\).

Interactive Example 1(c): Points on a Line

Using the vector equation from Example 1(b): \[ (x, y, z) = (1, 2, -3) + t(4, -5, 1) \] Let’s find points for different values of \(t\).

As you can see, for each value of t, we get a distinct point on the line. t=0 yields the starting point P0. t=1 moves us one unit along the v vector from P0. t=-1 moves us in the opposite direction. This is a direct application of vector addition and scalar multiplication.

Example 2: Plane from Rectangular Equation

Find vector and parametric equations for the plane \(x - y + 2z = 5\).

1. Solve for one variable: Solve for \(x\): \(x = 5 + y - 2z\)

2. Introduce parameters: Let \(y = t_1\) and \(z = t_2\).

3. Parametric equations:

   \(x = 5 + t_1 - 2t_2\)

   \(y = t_1\)

   \(z = t_2\)

4. Vector Equation: Rewrite in vector form:

   \((x, y, z) = (5 + t_1 - 2t_2, t_1, t_2)\)

   Split into components:

   \((x, y, z) = (5, 0, 0) + t_1(1, 1, 0) + t_2(-2, 0, 1)\)

Important

Note that we could have chosen to solve for \(y\) or \(z\) instead, leading to different parametric forms, but representing the same geometric plane.

This example demonstrates how to convert a standard Cartesian equation of a plane into its vector and parametric forms. The choice of which variable to solve for is arbitrary, but it affects the specific vectors v1 and v2 you obtain. However, the plane itself, the geometric object, remains the same. This transformation is useful in computer graphics, for example, where you might need to define a surface for rendering from an implicit equation.

Interactive Example 2: Plane Visualization

Visualize the plane \(x - y + 2z = 5\).

This visualization allows you to rotate and explore the plane defined by x - y + 2z = 5. Notice how it extends infinitely in all directions. In control systems, the solution space of certain linear equations can represent a constraint surface within which system states must operate.

Lines Through Two Points in \(R^n\)

A line can also be defined by two distinct points \(\mathbf{x}_0\) and \(\mathbf{x}_1\).

The line determined by \(\mathbf{x}_0\) and \(\mathbf{x}_1\) is parallel to the vector \(\mathbf{v} = \mathbf{x}_1 - \mathbf{x}_0\).

Using the general line equation \(\mathbf{x} = \mathbf{x}_0 + t\mathbf{v}\): \[ \mathbf{x} = \mathbf{x}_0 + t(\mathbf{x}_1 - \mathbf{x}_0) \quad \text{(9)} \]

This can be rewritten as: \[ \mathbf{x} = (1 - t)\mathbf{x}_0 + t\mathbf{x}_1 \quad \text{(10)} \]

These are the two-point vector equations of a line.

Figure 3.4.5

This formulation is very intuitive. If you have two points, you can find the direction vector between them. Then, any point on the line can be seen as starting at one point and moving along that direction. The form (1-t)x0 + t*x1 is particularly elegant as it clearly shows the weighted average of the two points, which is useful in interpolation.

Interactive: Line Through Two Points

Find the vector and parametric equations for a line passing through two given points.

Tip

Enter coordinates for \(\mathbf{x}_0\) and \(\mathbf{x}_1\) to generate the line equations and visualize it.

This interactive tool calculates both the vector and parametric forms and plots the line. You can easily change the input points x0_point and x1_point to see how the line adjusts. This is highly practical in engineering, for example, when defining paths for robots or data interpolation.

Line Segments

A line segment is a portion of a line between two specific points.

If \(\mathbf{x}_0\) and \(\mathbf{x}_1\) are vectors in \(R^n\), the equation: \[ \mathbf{x} = \mathbf{x}_0 + t(\mathbf{x}_1 - \mathbf{x}_0) \quad (0 \leq t \leq 1) \quad \text{(13)} \] defines the line segment from \(\mathbf{x}_0\) to \(\mathbf{x}_1\).

Equivalently: \[ \mathbf{x} = (1 - t)\mathbf{x}_0 + t\mathbf{x}_1 \quad (0 \leq t \leq 1) \quad \text{(14)} \]

• When \(t=0\), \(\mathbf{x} = \mathbf{x}_0\).
• When \(t=1\), \(\mathbf{x} = \mathbf{x}_1\).
• For \(0 < t < 1\), \(\mathbf{x}\) lies between \(\mathbf{x}_0\) and \(\mathbf{x}_1\).

Restricting the parameter \(t\) to the interval [0, 1] transforms a full line into a line segment. This is extremely important in computer graphics for drawing edges, in robotics for defining specific movement trajectories, or in signal processing for windowing functions. The form (1-t)x0 + t*x1 is often called a linear interpolation formula.

Interactive Line Segment

Visualize a line segment between two points and explore intermediate points.

Tip

Adjust the value of t_slider to see how a point moves along the segment. Change x0_coords and x1_coords to define a new segment.

viewof x0_x = Inputs.number([-10, 10], {label: "x0 (x-coor):", step: 1, value: 1})
viewof x0_y = Inputs.number([-10, 10], {label: "x0 (y-coordinate):", step: 1, value: -3})
viewof x1_x = Inputs.number([-10, 10], {label: "x1 (x-coordinate):", step: 1, value: 5})
viewof x1_y = Inputs.number([-10, 10], {label: "x1 (y-coordinate):", step: 1, value: 6})

// Recombine into coordinate arrays
x0_coords = [x0_x, x0_y]
x1_coords = [x1_x, x1_y]

// Create a slider for parameter t
viewof t_slider = Inputs.range([0, 1], {step: 0.01, value: 0.5, label: "Parameter t"})

// Calculate the point on the segment
current_point = [
  (1 - t_slider) * x0_coords[0] + t_slider * x1_coords[0],
  (1 - t_slider) * x0_coords[1] + t_slider * x1_coords[1]
]

// Create the plot using Observable Plot
Plot.plot({
  title: "Line Segment Visualization",
  x: {domain: [-6, 10], label: "X-axis"},
  y: {domain: [-6, 10], label: "Y-axis"},
  grid: true,
  marks: [
    Plot.line([[x0_coords[0], x1_coords[0]], [x0_coords[1], x1_coords[1]]], {stroke: "lightgray", strokeWidth: 3}),
    Plot.dot([x0_coords], {r: 8, fill: "red", title: `x0: (${x0_coords[0]}, ${x0_coords[1]})`}),
    Plot.dot([x1_coords], {r: 8, fill: "blue", title: `x1: (${x1_coords[0]}, ${x1_coords[1]})`}),
    Plot.dot([current_point], {r: 10, fill: "green", title: `Current Point (t=${t_slider.toFixed(2)}): (${current_point[0].toFixed(2)}, ${current_point[1].toFixed(2)})`})
  ]
})

This Observable.js visualization allows direct manipulation of the t parameter. You can drag the slider to see the point x move from x0 to x1. This is a visual representation of linear interpolation, a core concept in various ECE applications, such as digital signal processing for creating intermediate samples or in computer graphics for blending colors or positions.

Dot Product Form of a Linear System

Linear equations can be expressed concisely using dot products.

A linear equation \(a_1x_1 + \dots + a_nx_n = b\) can be written as: \[ \mathbf{a} \cdot \mathbf{x} = b \quad \text{(17)} \] where \(\mathbf{a} = (a_1, \dots, a_n)\) and \(\mathbf{x} = (x_1, \dots, x_n)\).

For a homogeneous equation (\(\mathbf{b}=0\)): \[ \mathbf{a} \cdot \mathbf{x} = 0 \quad \text{(18)} \] This implies that every solution vector \(\mathbf{x}\) is orthogonal to the coefficient vector \(\mathbf{a}\).

Dot Product Form of a Linear System

Homogeneous System \(A\mathbf{x} = \mathbf{0}\)

If \(A\) has row vectors \(\mathbf{r}_1, \dots, \mathbf{r}_m\), the system becomes: \[ \begin{array}{r}{\mathbf{r}_{1}\cdot \mathbf{x} = \mathbf{0}}\\ {\mathbf{r}_{2}\cdot \mathbf{x} = \mathbf{0}}\\ \vdots \quad \vdots \\ {\mathbf{r}_{m}\cdot \mathbf{x} = \mathbf{0}} \end{array} \quad \text{(19)} \] Theorem 3.4.3: The solution set of \(A\mathbf{x}=\mathbf{0}\) consists of all vectors in \(R^n\) that are orthogonal to every row vector of \(A\).

The dot product notation simplifies the representation of linear equations and systems. The geometric interpretation of a . x = 0 is crucial: x lies in the space orthogonal to a. When we extend this to a system Ax = 0, the solution space is orthogonal to all row vectors of A. This concept is fundamental in understanding null spaces, which are vital in signal processing for noise reduction and in control theory for system stability analysis.

Interactive Orthogonality Check

Verify the orthogonality between a solution vector and a row vector.

Tip

Change the solution_vector or row_vector to test different scenarios. A dot product of zero indicates orthogonality.

This interactive code verifies the geometric property of orthogonality. It’s a direct numerical check of Theorem 3.4.3. Notice how the first calculation, using a derived solution vector, correctly yields a dot product of zero. This has significant implications in signal processing for designing filters that eliminate certain signal components, or in error correction codes where orthogonal vectors represent distinct codewords.

Relationship Between \(A\mathbf{x}=\mathbf{0}\) and \(A\mathbf{x}=\mathbf{b}\)

The solutions of a nonhomogeneous system are a translation of the solutions of its corresponding homogeneous system.

Consider a consistent linear system \(A\mathbf{x} = \mathbf{b}\).

Let \(\mathbf{x}_p\) be any particular solution to \(A\mathbf{x} = \mathbf{b}\).

Let \(\mathbf{x}_h\) be the general solution to the homogeneous system \(A\mathbf{x} = \mathbf{0}\).

Theorem 3.4.4: The general solution of \(A\mathbf{x} = \mathbf{b}\) is given by: \[ \mathbf{x} = \mathbf{x}_p + \mathbf{x}_h \]

This means the solution set of \(A\mathbf{x}=\mathbf{b}\) is a translation of the solution space of \(A\mathbf{x}=\mathbf{0}\) by the particular solution \(\mathbf{x}_p\).

Figure 3.4.7

This theorem is incredibly powerful. It tells us that to find all solutions to a nonhomogeneous system, we just need one specific solution and then add all possible solutions to the homogeneous system. Geometrically, if the homogeneous solution set is a plane through the origin, the nonhomogeneous solution set is the same plane, just shifted to pass through xp. This concept is vital in solving differential equations, control system responses (forced vs. natural response), and understanding the structure of solution spaces in general.

Interactive: Solution Space Translation

Visualize how the solution set of \(A\mathbf{x} = \mathbf{b}\) is a translated version of the solution space of \(A\mathbf{x} = \mathbf{0}\).

Note

Here, we simulate a 1D solution space (a line) in 2D for simplicity. The homogeneous solution is a line through the origin. The nonhomogeneous solution is the same line, translated by \(\mathbf{x}_p\).

This plot vividly demonstrates the translation property. The blue line represents the solution space of Ax=0, always passing through the origin. The dashed green line represents the solution space of Ax=b, which is simply the blue line shifted by the particular solution xp (the red point). This visualization reinforces the idea that the structure of the homogeneous solution space determines the “shape” of the general solution, while xp merely positions it in space. This is analogous to the concept of steady-state and transient responses in circuit analysis.

Conclusion: Key Takeaways

• Vector & Parametric Equations: Powerful tools to describe lines and planes using points and direction vectors.
• Generalization to \(R^n\): These geometric concepts extend directly to higher dimensions, crucial for ECE applications.
• Orthogonality in Linear Systems: Solution vectors of \(A\mathbf{x}=\mathbf{0}\) are orthogonal to the row vectors of \(A\).
• Solution Space Translation: The solution set of \(A\mathbf{x}=\mathbf{b}\) is a translation of the solution space of \(A\mathbf{x}=\mathbf{0}\).

Important

Understanding the geometry behind linear systems is essential for intuitive problem-solving and deeper insights in fields like signal processing, control systems, and machine learning.

To wrap up, remember that linear algebra isn’t just about symbols and numbers; it’s about geometric relationships. These geometric interpretations provide intuition, help visualize complex data, and are directly applicable in many ECE domains. From designing filters to understanding system stability, these geometric insights are invaluable. Keep practicing these concepts and visualizing them!