Linear Algebra

4.1 Real Vector Spaces

Imron Rosyadi

Real Vector Spaces: Beyond \(R^n\)

Introduction: Extending the Vector Concept

In this section, we extend the familiar concept of a vector beyond \(R^n\). We use the fundamental properties of vectors in \(R^n\) as axioms. These axioms define what it means for a set of objects to “behave like” vectors.

We’re moving from concrete examples like 2D or 3D vectors to a more abstract, yet powerful, definition. This abstraction allows us to apply linear algebra tools to a much wider range of mathematical objects encountered in engineering. Think about signals, images, or even control system states as vectors.

What is a Vector Space?

A vector space is a nonempty set \(V\) of objects, called vectors, on which two operations are defined:

1. Addition: For any \(\mathbf{u}, \mathbf{v}\) in \(V\), their sum \(\mathbf{u} + \mathbf{v}\) is also in \(V\).
2. Scalar Multiplication: For any scalar \(k\) and \(\mathbf{u}\) in \(V\), the scalar multiple \(k\mathbf{u}\) is also in \(V\).

If these operations satisfy ten axioms, then \(V\) is called a vector space. Scalars will primarily be real numbers, defining a real vector space.

Note

In ECE, scalars often represent physical quantities like gain, amplitude, or time constants. The “objects” can be diverse: voltages, currents, signals, images, or even matrices.

Emphasize that the definition is abstract. The “vectors” don’t have to be arrows in space. They could be functions, matrices, or sequences. The key is that they obey these ten rules. We’ll focus on real vector spaces for now, as they are most common in introductory ECE applications.

The Ten Vector Space Axioms (Part 1)

Let \(\mathbf{u}\), \(\mathbf{v}\), \(\mathbf{w}\) be objects in \(V\), and \(k\), \(m\) be scalars.

Closure & Commutativity

1. Closure under Addition: If \(\mathbf{u}\) and \(\mathbf{v}\) are in \(V\), then \(\mathbf{u} + \mathbf{v}\) is in \(V\).
2. Commutativity: \(\mathbf{u} + \mathbf{v} = \mathbf{v} + \mathbf{u}\).
3. Associativity: \(\mathbf{u} + (\mathbf{v} + \mathbf{w}) = (\mathbf{u} + \mathbf{v}) + \mathbf{w}\).

Identity & Inverse

4. Zero Vector: There is a zero vector \(\mathbf{0}\) in \(V\) such that \(\mathbf{0} + \mathbf{u} = \mathbf{u} + \mathbf{0} = \mathbf{u}\) for all \(\mathbf{u}\) in \(V\).
5. Negative Vector: For each \(\mathbf{u}\) in \(V\), there is a negative vector \(-\mathbf{u}\) in \(V\) such that \(\mathbf{u} + (-\mathbf{u}) = (-\mathbf{u}) + \mathbf{u} = \mathbf{0}\).

These first five axioms deal with the addition operation. Closure means that adding two vectors together always results in another vector within the same set. Commutativity means order doesn’t matter for addition. Associativity means grouping doesn’t matter. The zero vector is like the additive identity, and the negative vector is the additive inverse. These might seem obvious for \(R^n\), but for other types of vectors, they need explicit verification.

The Ten Vector Space Axioms (Part 2)

Let \(\mathbf{u}\), \(\mathbf{v}\), \(\mathbf{w}\) be objects in \(V\), and \(k\), \(m\) be scalars.

Scalar Multiplication Properties

6. Closure under Scalar Multiplication: If \(k\) is any scalar and \(\mathbf{u}\) is in \(V\), then \(k\mathbf{u}\) is in \(V\).
7. Distributivity (Scalar over Vector Sum): \(k(\mathbf{u} + \mathbf{v}) = k\mathbf{u} + k\mathbf{v}\).
8. Distributivity (Scalar Sum over Vector): \((k + m)\mathbf{u} = k\mathbf{u} + m\mathbf{u}\).

Compatibility & Identity

9. Associativity of Scalar Multiplication: \(k(m\mathbf{u}) = (km)(\mathbf{u})\).
10. Scalar Identity: \(1\mathbf{u} = \mathbf{u}\).

Important

Axioms 1 and 6 (closure) are crucial. If these fail, the set cannot be a vector space, as the operations take us outside the defined set \(V\).

The remaining five axioms deal with scalar multiplication and how it interacts with vector addition. Again, these are intuitive for \(R^n\), but need to be proven for other vector candidates. Axiom 10, the scalar identity, ensures that multiplying by 1 leaves the vector unchanged, which is a fundamental requirement for the “scalar” concept.

How to Verify a Vector Space

To show that a set with two operations is a vector space, follow these four steps:

1. Identify \(V\): Clearly define the set of objects that will be called “vectors”.
2. Identify Operations: Clearly define the addition and scalar multiplication operations on \(V\).
3. Verify Closure (Axioms 1 & 6):
   • Adding two vectors in \(V\) must result in a vector in \(V\).
   • Multiplying a vector in \(V\) by a scalar must result in a vector in \(V\).
4. Confirm Remaining Axioms (Axioms 2, 3, 4, 5, 7, 8, 9, 10):
   • Prove that the other eight axioms hold true for the defined operations.

This systematic approach is essential. Don’t jump straight to proving axiom 10 without first defining the set and its operations. Closure axioms are often the easiest to check first and can quickly rule out a set from being a vector space.

Illustration of Vector Space Properties

Let’s visualize the structure of a vector space.

graph TD
    A[Set V of Objects] --> B{Operations Defined?};
    B -- Yes --> C(Vector Addition: +);
    B -- Yes --> D(Scalar Multiplication: .);

    C --> E{Axioms 1-5 Satisfied?};
    D --> F{Axioms 6-10 Satisfied?};

    E -- Yes --> G(Addition Properties Hold);
    F -- Yes --> H(Scalar Mult. Properties Hold);

    G & H --> I{All 10 Axioms Satisfied?};
    I -- Yes --> J[V is a Vector Space];
    I -- No --> K[V is NOT a Vector Space];

    subgraph Axioms
        E;F;
    end
    style J fill:#cef,stroke:#3c3,stroke-width:2px;
    style K fill:#fcc,stroke:#c33,stroke-width:2px;

This diagram visually represents the logical flow for determining if a given set and its operations form a vector space. It emphasizes the critical role of the ten axioms.

Example 1: The Zero Vector Space

Let \(V\) consist of a single object, denoted by \(\mathbf{0}\). Define operations:

\[ \mathbf{0} + \mathbf{0} = \mathbf{0} \text{ and } k\mathbf{0} = \mathbf{0} \]

for all scalars \(k\).

Verification:

• Axiom 1 (Closure under Addition): \(\mathbf{0} + \mathbf{0} = \mathbf{0}\), which is in \(V\). (Holds)
• Axiom 6 (Closure under Scalar Multiplication): \(k\mathbf{0} = \mathbf{0}\), which is in \(V\). (Holds)
• All other axioms are trivially satisfied because there’s only one object, the zero vector itself.

This is called the zero vector space.

This is the simplest possible vector space. It might seem trivial, but it’s a valid example and confirms that the axioms can be satisfied even in the most basic scenarios. It highlights that the zero vector is a fundamental requirement.

Example 2: \(R^n\) is a Vector Space

Let \(V = R^n\), the set of all \(n\)-tuples of real numbers. Define vector space operations as the usual operations of addition and scalar multiplication for \(n\)-tuples:

\[ \mathbf{u} + \mathbf{v} = (u_1, u_2, \ldots, u_n) + (v_1, v_2, \ldots, v_n) = (u_1+v_1, u_2+v_2, \ldots, u_n+v_n) \]

\[ k\mathbf{u} = (k u_1, k u_2, \ldots, k u_n) \]

Verification:

• Closure (Axioms 1 & 6): The operations produce \(n\)-tuples, so they are closed. (Holds)
• Remaining Axioms: These operations are precisely what the axioms were based on. They all hold true.

Tip

In ECE, \(R^n\) is ubiquitous! It represents signal samples, state vectors in control systems, pixel intensities in images, or coefficients of polynomials.

This is our foundational example. The axioms were literally derived from the properties of \(R^n\). So, it’s not surprising that \(R^n\) is indeed a vector space. This gives us confidence in the definition.

Interactive Example: Vector Addition in \(R^2\)

Adjust the components of vectors \(\mathbf{u}\) and \(\mathbf{v}\) to see their sum.

viewof u_x = Inputs.range([-5, 5], {label: "u_x", step: 0.1, value: 2})
viewof u_y = Inputs.range([-5, 5], {label: "u_y", step: 0.1, value: 3})
viewof v_x = Inputs.range([-5, 5], {label: "v_x", step: 0.1, value: 1})
viewof v_y = Inputs.range([-5, 5], {label: "v_y", step: 0.1, value: -2})

This interactive plot helps students visualize vector addition in \(R^2\). They can manipulate the components and see how the resultant vector changes, reinforcing the geometric interpretation of vector addition.

Example 3: The Vector Space of Infinite Sequences

Let \(V\) be the set of objects of the form \(\mathbf{u} = (u_1, u_2, \ldots, u_n, \ldots)\), an infinite sequence of real numbers. Define addition and scalar multiplication componentwise:

\[ \mathbf{u} + \mathbf{v} = (u_1+v_1, u_2+v_2, \ldots, u_n+v_n, \ldots) \]

\[ k\mathbf{u} = (k u_1, k u_2, \ldots, k u_n, \ldots) \]

This vector space is denoted by \(R^\infty\).

Tip

ECE Application: In signal processing, a discrete-time signal of infinite duration can be represented as an infinite sequence. Adding signals or scaling them (e.g., amplification) are common operations.

This example shows that vectors can have infinitely many components. This is very relevant in ECE, especially in digital signal processing where signals are often represented as sequences of values. The image illustrates how a continuous signal is converted into a discrete sequence, which can then be treated as a vector in \(R^\infty\).

Example 4: The Vector Space of \(2 \times 2\) Matrices

Let \(V\) be the set of \(2 \times 2\) matrices with real entries. We treat these matrices as our “vectors”. The operations are standard matrix addition and scalar multiplication.

\[ \mathbf{u} = \left[ \begin{array}{ll}u_{11} & u_{12} \\ u_{21} & u_{22} \end{array} \right], \quad \mathbf{v} = \left[ \begin{array}{ll}v_{11} & v_{12} \\ v_{21} & v_{22} \end{array} \right] \]

Tip

ECE Application: Matrices are fundamental in ECE for representing linear transformations, systems of equations (e.g., circuit analysis), and state-space models of dynamic systems.

This might be confusing at first because rows and columns of matrices are themselves vectors. However, here we consider the entire matrix as a single vector in the vector space \(M_{22}\). This is a crucial distinction. We are looking at the properties of the matrix operations as they apply to the matrix as a whole.

Example 4: \(2 \times 2\) Matrices - Operations

Addition:

\[ \mathbf{u} + \mathbf{v} = \left[ \begin{array}{ll}u_{11} & u_{12} \\ u_{21} & u_{22} \end{array} \right] + \left[ \begin{array}{ll}v_{11} & v_{12} \\ v_{21} & v_{22} \end{array} \right] = \left[ \begin{array}{ll}u_{11} + v_{11} & u_{12} + v_{12} \\ u_{21} + v_{21} & u_{22} + v_{22} \end{array} \right] \tag{1} \]

Scalar Multiplication:

\[ k\mathbf{u} = k\left[ \begin{array}{ll}u_{11} & u_{12} \\ u_{21} & u_{22} \end{array} \right] = \left[ \begin{array}{ll}k u_{11} & k u_{12} \\ k u_{21} & k u_{22} \end{array} \right] \]

Notice that Equation (1) involves three different addition operations: vector addition on the left, matrix addition in the middle, and numerical addition on the right. This highlights the abstraction.

Example 4: \(2 \times 2\) Matrices - Axiom Verification

The set \(V\) is closed under addition and scalar multiplication because the operations produce \(2 \times 2\) matrices.

Axiom 4 (Zero Vector): The zero vector for \(V\) is the \(2 \times 2\) zero matrix:

\[ \mathbf{0} = \left[ \begin{array}{ll}0 & 0 \\ 0 & 0 \end{array} \right] \]

Then \(\mathbf{0} + \mathbf{u} = \mathbf{u}\).

Axiom 5 (Negative Vector): The negative of \(\mathbf{u}\) is:

\[ -\mathbf{u} = \left[ \begin{array}{ll} - u_{11} & -u_{12} \\ -u_{21} & -u_{22} \end{array} \right] \]

Then \(\mathbf{u} + (-\mathbf{u}) = \mathbf{0}\).

Axiom 10 (Scalar Identity):

\[ 1\mathbf{u} = 1\left[ \begin{array}{ll}u_{11} & u_{12} \\ u_{21} & u_{22} \end{array} \right] = \left[ \begin{array}{ll}u_{11} & u_{12} \\ u_{21} & u_{22} \end{array} \right] = \mathbf{u} \]

The other axioms follow from properties of real numbers and matrix arithmetic.

This verification demonstrates how to formally check the axioms. It’s important to explicitly state what the zero vector and negative vector are in this specific context.

Example 5: The Vector Space of \(m \times n\) Matrices

The set \(V\) of all \(m \times n\) matrices with real entries, with the usual matrix operations of addition and scalar multiplication, is a vector space. This vector space is denoted by \(M_{mn}\).

• Example 4 (\(M_{22}\)) is a special case.
• The proof follows the same logic: closure, existence of zero matrix (all zeros), negative matrix (all elements negated), and other axioms derived from real number properties.

This generalizes the previous example. It’s important for students to recognize that the principles apply universally to matrices of any compatible dimensions. This is especially relevant in ECE for systems with multiple inputs/outputs or for representing complex data structures.

Example 6: The Vector Space of Real-Valued Functions

Let \(V\) be the set of real-valued functions defined at each \(x\) in the interval \((-\infty, \infty)\). If \(\mathbf{f} = f(x)\) and \(\mathbf{g} = g(x)\) are two functions in \(V\), and \(k\) is any scalar:

Addition:

\[ (\mathbf{f}+\mathbf{g})(x) = f(x) + g(x) \]

Scalar Multiplication:

\[ (k\mathbf{f})(x) = k f(x) \]

This vector space is denoted by \(F(-\infty, \infty)\).

Tip

ECE Application: Signals and systems theory heavily relies on functions as vectors. Superposition (linear combination) of signals is a direct application of vector space properties.

Here, the “components” of the functions are their values at each point \(x\). So, adding functions means adding their values at each point, and scalar multiplying a function means scaling its value at each point. This is analogous to component-wise operations in \(R^n\) or \(R^\infty\).

Example 6: Real-Valued Functions - Verification

Axioms 1 & 6 (Closure):

If \(f(x)\) and \(g(x)\) are defined for all \(x\), then \(f(x)+g(x)\) and \(k f(x)\) are also defined for all \(x\). (Holds)

Axiom 4 (Zero Vector):

The zero vector is the function \(\mathbf{0}(x) = 0\) for all \(x\). \(\mathbf{0}(x) + f(x) = 0 + f(x) = f(x)\).

Axiom 5 (Negative Vector):

The negative of \(f(x)\) is the function \((-\mathbf{f})(x) = -f(x)\). \(f(x) + (-f(x)) = 0 = \mathbf{0}(x)\).

Remaining Axioms:

Follow from properties of real numbers. For example, for Axiom 2: \((\mathbf{f}+\mathbf{g})(x) = f(x)+g(x) = g(x)+f(x) = (\mathbf{g}+\mathbf{f})(x)\).

The graph visually reinforces the concepts of function addition, scalar multiplication, and the negative of a function. This is highly relevant in ECE for understanding signal manipulation, such as combining multiple sensor inputs or amplifying a signal.

When a Set is NOT a Vector Space

It’s crucial to understand that not every set with defined addition and scalar multiplication operations forms a vector space. One or more axioms might fail.

Warning

Even if operations seem “normal,” careful verification of ALL ten axioms is required. A common pitfall is assuming closure if the operations are defined.

Example: The set of \(n\)-tuples with positive components, using standard \(R^n\) operations, is NOT a vector space.

• If \(\mathbf{u} = (1, 2)\) (positive components), and \(k = -1\).
• Then \(k\mathbf{u} = (-1, -2)\), which has negative components.
• This result is not in the set of \(n\)-tuples with positive components.
• Axiom 6 (Closure under Scalar Multiplication) fails!

This example highlights that closure axioms (1 and 6) are often the first to fail. If the operations take you out of the set, it’s immediately not a vector space. This is a simple but effective way to weed out non-vector spaces.

Example 7: \(R^2\) with Modified Scalar Multiplication

Let \(V = R^2\).

Define addition as standard: \(\mathbf{u} + \mathbf{v} = (u_1+v_1, u_2+v_2)\).

Define scalar multiplication as unusual: \(k\mathbf{u} = (k u_1, 0)\).

Let’s check Axiom 10: \(1\mathbf{u} = \mathbf{u}\).

If \(\mathbf{u} = (u_1, u_2)\) with \(u_2 \neq 0\):

\[ 1\mathbf{u} = 1(u_1, u_2) = (1 \cdot u_1, 0) = (u_1, 0) \]

This result \((u_1, 0)\) is not equal to \(\mathbf{u} = (u_1, u_2)\) if \(u_2 \neq 0\).

Therefore, Axiom 10 fails, and \(V\) is not a vector space with these operations.

This example demonstrates that even if 9 out of 10 axioms hold, if just one fails, it’s not a vector space. The modified scalar multiplication forces the second component to zero, breaking the identity property for any vector not already on the x-axis.

Example 8: An Unusual Vector Space

Let \(V\) be the set of positive real numbers.

Let \(\mathbf{u} = u\) and \(\mathbf{v} = v\) be any “vectors” (positive real numbers) in \(V\).

Let \(k\) be any scalar.

Define the operations on \(V\) as:

• Vector Addition: \(\mathbf{u} + \mathbf{v} = u \cdot v\) (numerical multiplication).
• Scalar Multiplication: \(k\mathbf{u} = u^k\) (numerical exponentiation).

Example: \(1 + 1 = 1 \cdot 1 = 1\).

\((2)(1) = 1^2 = 1\).

This is indeed a vector space!

This is a truly “unusual” vector space designed to challenge intuition. It shows how abstract the definition is. The ordinary meaning of addition and multiplication is replaced by new definitions, yet the ten axioms can still be satisfied. This is a great example to emphasize the formal nature of the definition, rather than relying on intuition from \(R^n\).

Example 8: Unusual Vector Space - Axiom Verification

Let’s confirm a few axioms:

Axiom 4 (Zero Vector):

The zero vector \(\mathbf{0}\) in this space is the number \(1\).

\(u + \mathbf{0} = u \cdot 1 = u\). (Holds)

Axiom 5 (Negative Vector):

The negative of a vector \(u\) is its reciprocal, \(1/u\).

\(u + (-u) = u \cdot (1/u) = 1 \, (= \mathbf{0})\). (Holds)

Axiom 7 (Distributivity): \(k(\mathbf{u} + \mathbf{v}) = k(u \cdot v) = (u \cdot v)^k = u^k \cdot v^k\).

Also, \(k\mathbf{u} + k\mathbf{v} = u^k + v^k = u^k \cdot v^k\).

So, \(k(\mathbf{u} + \mathbf{v}) = k\mathbf{u} + k\mathbf{v}\). (Holds)

The other axioms can also be verified.

This detailed verification of specific axioms for the “unusual” vector space further demonstrates the process. It’s important to remember that the symbols + and k u here represent the defined operations (\(uv\) and \(u^k\)), not standard numerical addition and multiplication.

Some Properties of Vectors

Once a set is confirmed to be a vector space, we can derive properties that hold for all vector spaces.

Theorem 4.1.1: Let \(V\) be a vector space, \(\mathbf{u}\) a vector in \(V\), and \(k\) a scalar; then:

1. \(0\mathbf{u} = \mathbf{0}\)

2. \(k\mathbf{0} = \mathbf{0}\)

3. \((-1)\mathbf{u} = -\mathbf{u}\)

4. If \(k\mathbf{u} = \mathbf{0}\), then \(k = 0\) or \(\mathbf{u} = \mathbf{0}\).

These properties, familiar from \(R^n\), are consequences of the ten axioms.

This theorem shows the power of abstraction. By proving these properties once for general vector spaces, they apply to all specific examples we’ve seen (R^n, matrices, functions, infinite sequences, even the unusual positive real numbers example). This is a core concept in mathematics and engineering: generalize once, apply everywhere.

Proof of Theorem 4.1.1 (Part a)

To prove: \(0\mathbf{u} = \mathbf{0}\).

We start with properties of scalars and axioms:

\[ \begin{array}{r l} \mathbf{0}\mathbf{u} + \mathbf{0}\mathbf{u} &= (0 + 0)\mathbf{u} & \text{[Axiom 8]} \\ &= \mathbf{0}\mathbf{u} & \text{[Property of the number 0]} \end{array} \]

Now, add the negative of \(\mathbf{0}\mathbf{u}\) (which exists by Axiom 5) to both sides:

\[ [\mathbf{0}\mathbf{u} + \mathbf{0}\mathbf{u}] + (-\mathbf{0}\mathbf{u}) = \mathbf{0}\mathbf{u} + (-\mathbf{0}\mathbf{u}) \]

Applying Axioms:

\[ \begin{array}{r l} \mathbf{0}\mathbf{u} + [\mathbf{0}\mathbf{u} + (-\mathbf{0}\mathbf{u})] &= \mathbf{0}\mathbf{u} + (-\mathbf{0}\mathbf{u}) & \text{[Axiom 3]} \\ \mathbf{0}\mathbf{u} + \mathbf{0} &= \mathbf{0} & \text{[Axiom 5]} \\ \mathbf{0}\mathbf{u} &= \mathbf{0} & \text{[Axiom 4]} \end{array} \]

This step-by-step proof, explicitly citing each axiom, reinforces the formal nature of vector space theory. It shows how familiar results are rigorously derived from the fundamental axioms.

Proof of Theorem 4.1.1 (Part c)

To prove: \((-1)\mathbf{u} = -\mathbf{u}\).

We must show that \(\mathbf{u} + (-1)\mathbf{u} = \mathbf{0}\) (by definition of \(-\mathbf{u}\) from Axiom 5).

\[ \begin{array}{r l} \mathbf{u} + (-1)\mathbf{u} &= 1\mathbf{u} + (-1)\mathbf{u} & \text{[Axiom 10]} \\ &= (1 + (-1))\mathbf{u} & \text{[Axiom 8]} \\ &= 0\mathbf{u} & \text{[Property of numbers]} \\ &= \mathbf{0} & \text{[Part (a) of this theorem]} \end{array} \]

Since \(\mathbf{u} + (-1)\mathbf{u} = \mathbf{0}\), it follows that \((-1)\mathbf{u}\) is the unique negative of \(\mathbf{u}\), so \((-1)\mathbf{u} = -\mathbf{u}\).

This proof demonstrates how previously proven parts of a theorem (like part ‘a’) can be used to prove subsequent parts. It also reinforces the definition of a negative vector.

Closing Observation & Engineering Relevance

The concept of a vector space unifies diverse mathematical objects:

• Geometric vectors (\(R^2, R^3\))
• Vectors in \(R^n\)
• Infinite sequences (\(R^\infty\))
• Matrices (\(M_{mn}\))
• Real-valued functions (\(F(-\infty, \infty)\))
• Even abstract constructions like the “unusual” vector space.

Whenever we prove a theorem about general vector spaces, it automatically applies to all these specific examples.

Important

Impact for ECE: This abstraction allows us to develop powerful tools and theories (like linear transformations, eigenvalues, eigenvectors) that are applicable across various ECE domains: signal processing, control systems, machine learning, circuit analysis, and more.

This final slide is crucial for tying everything together and highlighting the practical significance of this abstract concept for ECE students. It emphasizes that the effort put into understanding general vector spaces pays off by providing a unified framework for solving problems in many different areas of engineering.