Subspace

In Linear Algebra, linear subspace is also known as vector subspace, which is a subset of a larger vector space. A linear subspace is normally called as subspace when the context serves to distinguish it from other types of subspaces.

For all subspace \(U\), they have to satisfy the following 3 rules:

1. Additive identity \(0, u \in U\), and \(0+u = u+0\), \(0\) and \(u\) here are vectors.

2. Close under addition For all \(u, w \in U\), then \(u+w \in U\), \(u\) and \(w\) here are vectors.

3. Close under scalar multiplication For all \(u \in U\), \(k \in R\), then \(k*u \in U\), \(u\) is a vector and \(k\) is a constant.

Suppose that \(A\) is a \(m * n\) matrix that maps vectors in \(R^n\) to vectors in \(R^m\). The four fundamental subspaces associated with \(A\), two in \(R^n\) and two in \(R^n\).

\[ A=\left(\begin{array}{ll} a_{11} & \dots & a_{1n}\\ \dots & \dots & \dots\\ a_{m1} & \dots & a_{mn} \end{array}\right) = \left(\begin{array}{ll} 1 & 3 & 3 \\ 2 & 4 & 6 \\ 1 & 7 & 3 \end{array}\right) \in M_{m,n}(R) = M_{3,3}(R) \]

Column Space

The column space of A is the linear combination of all linearly independent non-zero columns in A (it works with or without Gaussian elimination), i.e. \(Ax\), subspace in \(R^m\).

So \(C(A)\) span \(\left\{\left(\begin{array}{c}a_{11} \\ \vdots \\ a_{m 1}\end{array}\right) \cdots\left(\begin{array}{c}a_{1 n} \\ \vdots \\ a_{m n}\end{array}\right) \right\}\) , and the rank of A is equal to the dimension of column space.

We apply Gaussian elimination on matrix A, then get \(A=\left(\begin{array}{lll} 1 & 3 & 3 \\ 2 & 4 & 6 \\ 1 & 7 & 3 \end{array}\right) {\longrightarrow}\left(\begin{array}{lll} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right)\).

Because column 1 and column 3 are not linearly independent, \(C(A)\) is the linear combination of column 2 and column 1 or column 2 and column 3, either of these two is correct.

If we choose column 1 and column 2, then \(C(A) = \left\{\left(\begin{array}{c}1 \\ 0 \\ 0\end{array}\right) , \left(\begin{array}{c}0 \\ 1 \\ 0\end{array}\right) \right\}\), and \(Rank(A) = dim(C(A)) = 2\).

Row Space

The row space of A is the linear combination of all non-zero rows in A after Gaussian elimination, i.e. \(A^{T}y\), subspace in \(R^n\).

So \(R(A) = C(A^T)\) span \(\left\{\left(a_{11} \ldots a_{1 n}\right) \cdots\left(a_{m 1} \cdots a_{m n}\right)\right\}\).

According to Gaussian elimination, we get \(A {\rightarrow}\left(\begin{array}{lll} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right)\), then row space of A is the non-zero rows (row 1 and row 2), \(R(A)=\left\{\left(1, 0, 3\right),\left(0, 1, 0\right)\right\}\).

Nullspace

The nullspace of A is the linear combination of all solution of \(Ax = 0\), subspace in \(R^n\).

According to Gaussian elimination, we get \(A {\rightarrow}\left(\begin{array}{lll} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right)\), then we get

\(\left(\begin{array}{lll}1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{array}\right) \cdot\left(\begin{array}{l}x_{1} \\ x_{2} \\ x_{3}\end{array}\right)=\left\{\begin{array}{c}x_{1}+3 x_{3}=0 \\ x_{2}=0\end{array} \rightarrow\left\{\begin{array}{c}x_{1}=-3 x_{3} \\ x_{2}=0 \\ x_{3} \in R\end{array} \rightarrow x=N(A)=\left(\begin{array}{c}-3 \\ 0 \\ 1\end{array}\right) \cdot x_{3}\right.\right.\)

So a basis of \(N(A)\) is \((-3, 0, 1 )\), and nullity of A: \(null(A) = dim(N(A)) = 1\).

Left Nullspace

The left nullspace of A is the linear combination of all solution of \(A^{T}y = 0\), subspace in \(R^m\).

We apply Gaussian elimination on matrix \(A^T\), then get \(A^T=\left(\begin{array}{lll} 1 & 2 & 1 \\ 3 & 4 & 7 \\ 3 & 6 & 3 \end{array}\right) {\longrightarrow}\left(\begin{array}{lll} 1 & 0 & 5 \\ 0 & 1 & -2 \\ 0 & 0 & 0 \end{array}\right)\)

\(\left(\begin{array}{lll}1 & 0 & 5 \\ 0 & 1 & -2 \\ 0 & 0 & 0\end{array}\right) \cdot\left(\begin{array}{l}y_{1} \\ y_{2} \\ y_{3}\end{array}\right)=\left\{\begin{array}{c}y_{1}+5 y_{3}=0 \\ y_{2}-2y_{3}=0\end{array} \rightarrow\left\{\begin{array}{c}y_{1}=-5 y_{3} \\ y_{2}=2 y_{3} \\ y_{3} \in R\end{array} \rightarrow y=N(A^T)=\left(\begin{array}{c}-5 \\ 2 \\ 1\end{array}\right) \cdot y_{3}\right.\right.\)  

So a basis of \(N(A^T)\) is \((-5, 2, 1 )\), and \(null(A^T) = dim(N(A^T)) = 1\).

Characteristic of these four subspaces

1. Orthogonal: Column space and left nullspace are orthogonal, row space and null space are orthogonal.
   

\(C(A) \perp N\left(A^{\top}\right) \rightarrow(1,2,1) \cdot\left(\begin{array}{c}-5 \\ 2 \\ 1\end{array}\right)=(3,4,7) \cdot\left(\begin{array}{c}-5 \\ 2 \\ 1\end{array}\right)=0\)

\(R(A) \perp N(A) \rightarrow(1,0,3) \cdot\left(\begin{array}{c}-3 \\ 0 \\ 1\end{array}\right)=(0,1,0) \cdot\left(\begin{array}{c}-3 \\ 0 \\ 1\end{array}\right)=0\)

2. Rank-nullity:

Rank + Nullity = Column Number

dim(Row Space) + dim(Left Nullsapce) = Row Number