Linear Algebra

4.4 Coordinates and Basis

Imron Rosyadi

Coordinates and Basis

A Foundation for Vector Space Understanding

Welcome to this session on Coordinates and Basis. This topic is fundamental in Linear Algebra, especially for ECE students, as it underpins many concepts in signal processing, control systems, and machine learning. Today, we’ll explore how coordinate systems generalize beyond the familiar Cartesian plane and how basis vectors provide a unique way to represent any vector in a given space. We’ll also see some practical Python examples.

The Essence of Coordinate Systems

Coordinate systems allow us to describe points and vectors numerically. While rectangular systems are common, they are not the only option.

Rectangular Coordinates

• Familiar \(x, y, z\) axes.
• Mutually perpendicular axes.
• Common in basic geometry.

Non-Rectangular Coordinates

• Axes not necessarily perpendicular.
• Still define unique positions.
• More flexible for various applications.

We’re all familiar with rectangular coordinate systems from calculus and physics. They use mutually perpendicular axes, like the x, y, and z axes, to assign unique coordinates to points. However, in linear algebra, we generalize this idea. We can define coordinate systems where the axes are not necessarily perpendicular, as shown in the figures you might have seen in textbooks. This flexibility is crucial for understanding more abstract vector spaces.

Coordinate Systems: From Geometry to Vectors

In linear algebra, coordinate systems are often defined using vectors. These vectors define the “directions” and “units” for each axis.

Figure 4.4.1: Rectangular coordinates.

Figure 4.4.2: Non-rectangular coordinates.

Here, you see the visual representation of both rectangular and non-rectangular coordinate systems. Notice how in the non-rectangular case, the axes are still straight lines passing through the origin, but they are not at 90-degree angles to each other. This is perfectly valid and useful in many scenarios.

Defining Coordinates with Vectors

We use vectors, often unit vectors, to identify positive directions. A point \(P\) is then represented by scalar coefficients.

\[ \overrightarrow{OP} = a\mathbf{u}_1 + b\mathbf{u}_2 \quad \mathrm{and} \quad \overrightarrow{OP} = a\mathbf{u}_1 + b\mathbf{u}_2 + c\mathbf{u}_3 \]

Figure 4.4.3: Coordinate systems defined by unit vectors.

In linear algebra, we formalize this by using vectors, \(\mathbf{u}_1, \mathbf{u}_2\), and so on, to define the coordinate axes. Any point or vector \(\overrightarrow{OP}\) can then be expressed as a linear combination of these defining vectors. The coefficients, \(a, b, c\), become the coordinates of the point relative to this vector-defined system. Initially, these vectors might be unit vectors like \(\mathbf{i}, \mathbf{j}, \mathbf{k}\).

Basis Vectors: Generalizing Coordinate Axes

To allow for more generality, we relax the unit vector requirement. The defining vectors, now called basis vectors, only need to be linearly independent.

• Directions: Established by the basis vectors.
• Spacing: Determined by the lengths of the basis vectors.
• Requirement: Basis vectors must be linearly independent.

Figure 4.4.4: Basis vectors determining directions and spacing.

The key insight here is that the vectors defining our coordinate system don’t necessarily have to be unit vectors, nor do they need to be orthogonal. The crucial property is that these “basis vectors” must be linearly independent. This ensures that each vector in the space has a unique representation. The directions of these vectors define the positive directions, and their lengths determine the scaling or spacing along those “axes.” This concept is powerful because it extends to abstract vector spaces beyond just geometric R^2 or R^3.

Basis for a Vector Space: Definition

A basis provides a fundamental set of building blocks for a vector space. Vector spaces can be finite-dimensional or infinite-dimensional.

DEFINITION 1

If \(S = \{\mathbf{v}_{1}, \mathbf{v}_{2}, \ldots , \mathbf{v}_{n}\}\) is a set of vectors in a finite-dimensional vector space \(V\), then \(S\) is called a basis for \(V\) if:

1. \(S\) spans \(V\).
2. \(S\) is linearly independent.

Now, let’s formalize what a “basis” means in the context of general vector spaces. A basis is a special set of vectors that satisfies two critical conditions: first, it must span the entire vector space, meaning every vector in the space can be formed by a linear combination of the basis vectors. Second, the vectors in the basis must be linearly independent, meaning no vector in the set can be expressed as a linear combination of the others. These two conditions together ensure that we have just enough vectors to describe the entire space, without any redundancy. This definition applies to finite-dimensional vector spaces, which are our primary focus.

Finite vs. Infinite-Dimensional Spaces

Vector spaces are categorized by whether they have a finite spanning set.

Finite-Dimensional

• There exists a finite set of vectors that spans \(V\).
• Examples: \(R^n\), \(P_n\) (polynomials of degree \(\le n\)), \(M_{mn}\) (matrices).

Infinite-Dimensional

• No finite set of vectors can span \(V\).
• Examples: \(P_\infty\) (all polynomials), \(R^\infty\) (infinite sequences), \(C(-\infty, \infty)\) (continuous functions).

The concept of a basis naturally leads to the idea of dimension. A vector space is finite-dimensional if we can find a finite set of vectors that spans it. Most of the spaces we deal with in ECE, like \(R^n\) for signals or state vectors, are finite-dimensional. However, some spaces, like the space of all continuous functions or infinite sequences, are infinite-dimensional because no finite set of vectors can possibly span them. For these, the concept of a basis still exists, but it involves infinite sets of vectors, which is usually covered in more advanced courses.

EXAMPLE 1: The Standard Basis for \(R^{n}\)

The most familiar basis for \(R^n\) is the standard basis.

The standard unit vectors: \[ \mathbf{e}_{1} = (1,0,0,\ldots ,0), \quad \mathbf{e}_{2} = (0,1,0,\ldots ,0), \ldots , \quad \mathbf{e}_{n} = (0,0,0,\ldots ,1) \]

For \(R^3\), the standard basis is: \[ \mathbf{i} = (1,0,0), \quad \mathbf{j} = (0,1,0), \quad \mathbf{k} = (0,0,1) \]

These vectors are linearly independent and span \(R^n\).

Let’s look at some concrete examples of bases. The standard basis for \(R^n\) is probably the one you’re most familiar with. For \(R^3\), these are the \(\mathbf{i}\), \(\mathbf{j}\), and \(\mathbf{k}\) vectors. They are clearly linearly independent, as you can’t form one from the others, and any vector in \(R^3\) can be uniquely expressed as a linear combination of them. This makes them a perfect basis.

Interactive Example: Standard Basis in \(R^3\)

Visualize the standard basis vectors and a vector formed by their linear combination.

Here’s an interactive Python example using matplotlib to visualize the standard basis vectors in \(R^3\). The red, green, and blue arrows represent \(\mathbf{e}_1\), \(\mathbf{e}_2\), and \(\mathbf{e}_3\) respectively. The black arrow shows a vector \(\mathbf{v}\) that is a linear combination of these basis vectors. You can see how \(\mathbf{v}\) is constructed by scaling and summing the basis vectors.

EXAMPLE 2: The Standard Basis for \(P_{n}\)

The vector space \(P_n\) consists of polynomials of degree \(n\) or less. The standard basis for \(P_n\) is \(S = \{1, x, x^{2}, \ldots , x^{n}\}\).

Let’s denote these polynomials as: \[ \mathbf{p}_{0} = 1, \quad \mathbf{p}_{1} = x, \quad \mathbf{p}_{2} = x^{2}, \ldots , \quad \mathbf{p}_{n} = x^{n} \]

These polynomials span \(P_n\) and are linearly independent, forming a basis.

Not all vector spaces are about geometric vectors. Consider \(P_n\), the space of polynomials of degree \(n\) or less. A basis for this space is simply the set of monomials \(\{1, x, x^2, \ldots, x^n\}\). Any polynomial of degree \(n\) or less can be written as a unique linear combination of these. For example, \(3x^2 - 2x + 1\) is a linear combination of \(1, x, x^2\). These are also linearly independent; you can’t create \(x^2\) from \(1\) and \(x\).

EXAMPLE 3: Another Basis for \(R^{3}\)

A vector space can have multiple bases. Show that \(\mathbf{v}_{1} = (1,2,1)\), \(\mathbf{v}_{2} = (2,9,0)\), and \(\mathbf{v}_{3} = (3,3,4)\) form a basis for \(R^{3}\).

To prove this, we need to show:

1. Linear Independence: The equation \(c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + c_{3}\mathbf{v}_{3} = \mathbf{0}\) has only the trivial solution (\(c_1=c_2=c_3=0\)).
2. Spanning: Any vector \(\mathbf{b} = (b_{1},b_{2},b_{3})\) in \(R^{3}\) can be expressed as \(c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + c_{3}\mathbf{v}_{3} = \mathbf{b}\).

Both conditions rely on the invertibility of the coefficient matrix \(A\).

\[ A={\left[\begin{array}{l l l}{1}&{2}&{3}\\ {2}&{9}&{3}\\ {1}&{0}&{4}\end{array}\right]} \]

If \(\det(A) \neq 0\), then \(A\) is invertible, implying both conditions are met.

It’s important to understand that a vector space can have many different bases. Here’s an example of a non-standard basis for \(R^3\). To prove that these three vectors form a basis, we need to verify two things: they are linearly independent, and they span \(R^3\). Both of these conditions can be checked by examining the determinant of the matrix formed by these vectors. If the determinant is non-zero, the matrix is invertible, which guarantees both linear independence and spanning for \(R^3\).

Interactive Example: Checking Basis for \(R^3\)

Let’s use Python to calculate the determinant of matrix \(A\).

Here, we use numpy to construct the matrix \(A\) from our given vectors and then calculate its determinant. As you can see, the determinant is -1, which is non-zero. This confirms that the vectors \(\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\) are indeed linearly independent and span \(R^3\), thus forming a valid basis. This is a powerful computational check.

EXAMPLE 4: The Standard Basis for \(M_{mn}\)

The vector space \(M_{mn}\) consists of \(m \times n\) matrices. For \(M_{22}\) ( \(2 \times 2\) matrices), the standard basis is:

\[ M_{1}={\left[\begin{array}{l l}{1}&{0}\\ {0}&{0}\end{array}\right]},\quad M_{2}={\left[\begin{array}{l l}{0}&{1}\\ {0}&{0}\end{array}\right]},\quad M_{3}={\left[\begin{array}{l l}{0}&{0}\\ {1}&{0}\end{array}\right]},\quad M_{4}={\left[\begin{array}{l l}{0}&{0}\\ {0}&{1}\end{array}\right]} \]

Any \(2 \times 2\) matrix \(B = \left[ \begin{array}{ll}a & b \\ c & d \end{array} \right]\) can be uniquely written as:

\[ B = aM_1 + bM_2 + cM_3 + dM_4 \]

These matrices are linearly independent and span \(M_{22}\).

Even matrices can form vector spaces, and similarly, they have bases. For \(M_{22}\), the space of \(2 \times 2\) matrices, the standard basis consists of four matrices, each with a single ‘1’ and zeros elsewhere. Any \(2 \times 2\) matrix can be expressed as a unique linear combination of these four basis matrices. This concept extends to \(M_{mn}\) for any \(m\) and \(n\).

The Zero Vector Space

The simplest vector space is \(V = \{\mathbf{0}\}\).

It is finite-dimensional, spanned by \(\mathbf{0}\).

However, \(\{\mathbf{0}\}\) is not linearly independent.

Tip

For convenience, the empty set \(\varnothing\) is defined as a basis for the zero vector space.

A special case is the zero vector space, containing only the zero vector. It’s finite-dimensional because it’s spanned by the zero vector itself. However, a set containing only the zero vector is not linearly independent because you can always write \(c \cdot \mathbf{0} = \mathbf{0}\) for any non-zero scalar \(c\). To maintain consistency with the definition of a basis, we conventionally define the empty set as the basis for the zero vector space.

EXAMPLE 5 & 6: Infinite-Dimensional Vector Space

The vector space of all polynomials, \(P_{\infty}\), is infinite-dimensional.

No finite set of polynomials can span \(P_{\infty}\).

Why?

• If \(S = \{\mathbf{p}_{1}, \ldots , \mathbf{p}_{r}\}\) spanned \(P_{\infty}\), there would be a maximum degree, say \(n\).
• Then \(x^{n+1}\) could not be expressed as a linear combination of polynomials in \(S\).
• This contradicts \(S\) spanning \(P_{\infty}\).

Other infinite-dimensional spaces:

• \(R^{\infty}\) (sequences)
• \(F(- \infty , \infty)\) (all real-valued functions)
• \(C(- \infty , \infty)\) (continuous functions)
• \(C^{m}(- \infty , \infty)\) (functions with \(m\) continuous derivatives)

Let’s quickly revisit infinite-dimensional spaces. The space of all polynomials, \(P_{\infty}\), is a classic example. If you had a finite set of polynomials, say up to degree \(n\), you could never form a polynomial of degree \(n+1\) from their linear combinations. This shows that no finite set can span \(P_{\infty}\). Similarly, spaces of functions or infinite sequences are also infinite-dimensional, playing crucial roles in advanced engineering topics like Fourier analysis and signal processing.

Coordinates Relative to a Basis

A basis allows us to assign unique coordinates to every vector in a space. This formalizes the idea of a “coordinate system” in abstract vector spaces.

THEOREM 4.4.1 Uniqueness of Basis Representation

If \(S = \{\mathbf{v}_{1}, \mathbf{v}_{2}, \ldots , \mathbf{v}_{n}\}\) is a basis for a vector space \(V\), then every vector \(\mathbf{v}\) in \(V\) can be expressed in the form \(\mathbf{v} = c_{1} \mathbf{v}_{1} + c_{2} \mathbf{v}_{2} + \dots + c_{n} \mathbf{v}_{n}\) in exactly one way.

The most important consequence of having a basis is that every vector in the space can be uniquely represented as a linear combination of the basis vectors. This is Theorem 4.4.1, and it’s fundamental. The uniqueness comes directly from the linear independence of the basis vectors. If there were two different ways to write a vector, their difference would be a non-trivial linear combination of basis vectors equal to zero, which contradicts linear independence.

Defining Coordinate Vectors

The coefficients in the unique linear combination are the coordinates.

DEFINITION 2

If \(S = \{\mathbf{v}_{1}, \mathbf{v}_{2}, \ldots , \mathbf{v}_{n}\}\) is a basis for a vector space \(V\), and \[ \mathbf{v} = c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + \dots + c_{n}\mathbf{v}_{n} \] is the expression for a vector \(\mathbf{v}\) in terms of the basis \(S\), then the scalars \(c_{1}, c_{2}, \ldots , c_{n}\) are called the coordinates of \(\mathbf{v}\) relative to the basis \(S\). The vector \((c_{1}, c_{2}, \ldots , c_{n})\) in \(R^{n}\) is called the coordinate vector of \(\mathbf{v}\) relative to \(S\); it is denoted by \[ (\mathbf{v})_{S} = (c_{1}, c_{2}, \ldots , c_{n}) \tag{6} \]

This leads us to the formal definition of coordinates relative to a basis. Once we have a basis, the scalar coefficients \(c_1, \ldots, c_n\) that uniquely express a vector \(\mathbf{v}\) are called its coordinates relative to that basis. We can then collect these coefficients into a coordinate vector, denoted \((\mathbf{v})_S\). This coordinate vector is an element of \(R^n\), effectively translating vectors from any \(n\)-dimensional vector space into the familiar \(R^n\).

Ordered Basis

The order of basis vectors matters for coordinate vectors.

Note

When we talk about coordinate vectors, we always assume an ordered basis.

Changing the order of basis vectors changes the coordinate vector.

For example, in \(R^2\), \((1,2)\) is not the same as \((2,1)\).

Figure 4.4.6: Correspondence between vectors in \(V\) and \(R^n\).

A crucial point about coordinate vectors is that the order of the basis vectors is critical. While a set of basis vectors is mathematically a set (where order doesn’t matter), when we form a coordinate vector, we implicitly assume an ordered basis. If you change the order of the basis vectors, the coordinate vector will change, even though the underlying vector in \(V\) remains the same. This one-to-one correspondence between vectors in \(V\) and coordinate vectors in \(R^n\) is what makes coordinate systems so powerful.

EXAMPLE 7: Coordinates Relative to the Standard Basis for \(R^{n}\)

For the standard basis \(S\) in \(R^n\), the coordinate vector is identical to the vector itself.

For \(\mathbf{v} = (a,b,c)\) in \(R^3\) and standard basis \(S = \{\mathbf{i},\mathbf{j},\mathbf{k}\}\): \[ \mathbf{v} = a\mathbf{i} + b\mathbf{j} + c\mathbf{k} \] The coordinate vector relative to \(S\) is: \[ (\mathbf{v})_{S} = (a,b,c) \] This means \((\mathbf{v})_{S} = \mathbf{v}\).

Let’s quickly re-examine the standard basis. For \(R^n\) with its standard basis, the coordinates of a vector are simply its components. So, the coordinate vector \((\mathbf{v})_S\) is identical to the vector \(\mathbf{v}\) itself. This is why we often don’t explicitly distinguish between a vector and its coordinates in \(R^n\) when working with the standard basis.

EXAMPLE 8: Coordinate Vectors Relative to Standard Bases

Coordinate vectors for polynomials and matrices are also straightforward with standard bases.

(a) Polynomial \(P_n\)

For \(\mathbf{p}(x) = c_{0} + c_{1}x + \dots +c_{n}x^{n}\) and standard basis \(S = \{1,x,\ldots ,x^{n}\}\), \[ (\mathbf{p})_{S} = (c_{0},c_{1},c_{2},\ldots ,c_{n}) \]

(b) Matrix \(M_{22}\)

For \(B = \left[ \begin{array}{ll}a & b \\ c & d \end{array} \right]\) and standard basis \(S = \{M_1, M_2, M_3, M_4\}\), \[ (B)_{S} = (a,b,c,d) \]

Similarly, for polynomials and matrices, when using their respective standard bases, the coordinate vector is simply the vector of coefficients or entries. For a polynomial, it’s the coefficients from constant term to highest degree. For a matrix, it’s the entries read in a specific order (e.g., row by row). These examples highlight how the abstract definition of a coordinate vector translates to concrete representations in different vector spaces.

EXAMPLE 9: Coordinates in \(R^{3}\) with a Non-Standard Basis

Using the basis \(S = \{\mathbf{v}_{1},\mathbf{v}_{2},\mathbf{v}_{3}\}\) from Example 3: \(\mathbf{v}_{1} = (1,2,1)\), \(\mathbf{v}_{2} = (2,9,0)\), \(\mathbf{v}_{3} = (3,3,4)\).

(a) Find \((\mathbf{v})_{S}\) for \(\mathbf{v} = (5, - 1,9)\).

We need \(c_{1},c_{2},c_{3}\) such that \(\mathbf{v} = c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + c_{3}\mathbf{v}_{3}\).

This leads to the system: \[ \begin{array}{r l} & {c_{1} + 2c_{2} + 3c_{3} = 5}\\ & {2c_{1} + 9c_{2} + 3c_{3} = -1}\\ & {c_{1}\qquad +4c_{3} = 9} \end{array} \]

Now for a more challenging example, let’s find the coordinates of a vector relative to the non-standard basis we saw earlier. Given \(\mathbf{v} = (5, -1, 9)\), we want to find the scalars \(c_1, c_2, c_3\) such that \(\mathbf{v}\) is a linear combination of \(\mathbf{v}_1, \mathbf{v}_2, \mathbf{v}_3\). This translates into solving a system of linear equations, where the unknowns are our coordinates.

Interactive Example: Finding Coordinates

Let’s solve the system to find the coordinates \((\mathbf{v})_{S}\).

The solution is \(c_{1} = 1, c_{2} = - 1, c_{3} = 2\). So, \((\mathbf{v})_{S} = (1, -1,2)\).

Using numpy.linalg.solve, we can efficiently find the coefficients \(c_1, c_2, c_3\). The output shows that \(c_1=1, c_2=-1, c_3=2\). This means the coordinate vector of \(\mathbf{v}\) relative to basis \(S\) is \((1, -1, 2)\). We can also verify this by reconstructing \(\mathbf{v}\) from these coordinates and the basis vectors to ensure it matches the original vector.

EXAMPLE 9: Coordinates in \(R^{3}\) (Continued)

(b) Find the vector \(\mathbf{v}\) in \(R^{3}\) whose coordinate vector relative to \(S\) is \((\mathbf{v})_{S} = (-1,3,2)\).

Here, we are given the coordinates and need to find the actual vector. Using the definition \(\mathbf{v} = c_{1}\mathbf{v}_{1} + c_{2}\mathbf{v}_{2} + c_{3}\mathbf{v}_{3}\):

\[ \begin{array}{r l} & {\mathbf{v} = (-1)\mathbf{v}_{1} + 3\mathbf{v}_{2} + 2\mathbf{v}_{3}}\\ & {\quad = (-1)(1,2,1) + 3(2,9,0) + 2(3,3,4)} \end{array} \]

Now, let’s reverse the process. If we’re given the coordinate vector, say \((-1, 3, 2)\) relative to basis \(S\), we can find the actual vector \(\mathbf{v}\) in \(R^3\). This is simpler; we just perform the linear combination using the given coefficients and the basis vectors.

Interactive Example: Reconstructing Vector from Coordinates

Let’s compute \(\mathbf{v}\) from its coordinate vector.

The resulting vector is \(\mathbf{v} = (11,31,7)\).

By plugging in the coordinates and the basis vectors, we compute the linear combination. The output shows that the reconstructed vector \(\mathbf{v}\) is \((11, 31, 7)\). This demonstrates the direct relationship between a vector and its coordinates once a basis is defined. This is a fundamental operation in many engineering applications, such as changing coordinate frames in robotics or signal transformations.

Summary

• Coordinate Systems: Generalize beyond rectangular axes using vectors.
• Basis: A set of linearly independent vectors that spans a vector space.
• Uniqueness: Every vector has a unique representation relative to a given basis.
• Coordinate Vector: The ordered set of scalars representing a vector in terms of a basis.
• Application: Crucial for understanding transformations, solving systems, and representing data in ECE.

To summarize, we’ve explored how coordinate systems are generalized in linear algebra through the concept of a basis. A basis is a fundamental set of building blocks that allows for the unique representation of any vector in a vector space. The coordinate vector provides a numerical “address” for a vector relative to a specific basis. Understanding these concepts is paramount for ECE, as it forms the backbone for topics like change of basis, linear transformations, and efficient data representation in various engineering fields.