Subspace

In Linear Algebra, linear subspace is also known as vector subspace, which is a subset of a larger vector space. A linear subspace is normally called as subspace when the context serves to distinguish it from other types of subspaces.

For all subspace $U$, they have to satisfy the following 3 rules:

(1) Additive identity
$0, u \in U$, and $0+u = u+0$, $0$ and $u$ here are vectors.

(2) Close under addition
For all $u, w \in U$, then $u+w \in U$, $u$ and $w$ here are vectors.

(3) Close under scalar multiplication
For all $u \in U$, $k \in R$, then $k*u \in U$, $u$ is a vector and $k$ is a constant.

Suppose that $A$ is a $m * n$ matrix that maps vectors in $R^n$ to vectors in $R^m$. The four fundamental subspaces associated with $A$, two in $R^n$ and two in $R^n$.

$A=\left(\begin{array}{ll} a_{11} & \dots & a_{1n}\\ \dots & \dots & \dots\\ a_{m1} & \dots & a_{mn} \end{array}\right) = \left(\begin{array}{ll} 1 & 3 & 3 \\ 2 & 4 & 6 \\ 1 & 7 & 3 \end{array}\right) \in M_{m,n}(R) = M_{3,3}(R)$

Column Space

The column space of A is the linear combination of all linearly independent non-zero columns in A (it works with or without Gaussian elimination), i.e. $Ax$, subspace in $R^m$.

So $C(A)$ span $\left\{\left(\begin{array}{c}a_{11} \\ \vdots \\ a_{m 1}\end{array}\right) \cdots\left(\begin{array}{c}a_{1 n} \\ \vdots \\ a_{m n}\end{array}\right) \right\}$ , and the rank of A is equal to the dimension of column space.

We apply Gaussian elimination on matrix A, then get $A=\left(\begin{array}{lll} 1 & 3 & 3 \\ 2 & 4 & 6 \\ 1 & 7 & 3 \end{array}\right) {\longrightarrow}\left(\begin{array}{lll} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right)$.

Because column 1 and column 3 are not linearly independent, $C(A)$ is the linear combination of column 2 and column 1 or column 2 and column 3, either of these two is correct.

If we choose column 1 and column 2, then $C(A) = \left\{\left(\begin{array}{c}1 \\ 0 \\ 0\end{array}\right) , \left(\begin{array}{c}0 \\ 1 \\ 0\end{array}\right) \right\}$, and $Rank(A) = dim(C(A)) = 2$.

Row Space

The row space of A is the linear combination of all non-zero rows in A after Gaussian elimination, i.e. $A^{T}y$, subspace in $R^n$.

So $R(A) = C(A^T)$ span $\left\{\left(a_{11} \ldots a_{1 n}\right) \cdots\left(a_{m 1} \cdots a_{m n}\right)\right\}$.

According to Gaussian elimination, we get $A {\rightarrow}\left(\begin{array}{lll} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right)$, then row space of A is the non-zero rows (row 1 and row 2), $R(A)=\left\{\left(1, 0, 3\right),\left(0, 1, 0\right)\right\}$.

Nullspace

The nullspace of A is the linear combination of all solution of $Ax = 0$, subspace in $R^n$.

According to Gaussian elimination, we get $A {\rightarrow}\left(\begin{array}{lll} 1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0 \end{array}\right)$, then we get \

$\left(\begin{array}{lll}1 & 0 & 3 \\ 0 & 1 & 0 \\ 0 & 0 & 0\end{array}\right) \cdot\left(\begin{array}{l}x_{1} \\ x_{2} \\ x_{3}\end{array}\right)=\left\{\begin{array}{c}x_{1}+3 x_{3}=0 \\ x_{2}=0\end{array} \rightarrow\left\{\begin{array}{c}x_{1}=-3 x_{3} \\ x_{2}=0 \\ x_{3} \in R\end{array} \rightarrow x=N(A)=\left(\begin{array}{c}-3 \\ 0 \\ 1\end{array}\right) \cdot x_{3}\right.\right.$ \

So a basis of $N(A)$ is $(-3, 0, 1 )$, and nullity of A: $null(A) = dim(N(A)) = 1$.

Left Nullspace

The left nullspace of A is the linear combination of all solution of $A^{T}y = 0$, subspace in $R^m$.

We apply Gaussian elimination on matrix $A^T$, then get $A^T=\left(\begin{array}{lll} 1 & 2 & 1 \\ 3 & 4 & 7 \\ 3 & 6 & 3 \end{array}\right) {\longrightarrow}\left(\begin{array}{lll} 1 & 0 & 5 \\ 0 & 1 & -2 \\ 0 & 0 & 0 \end{array}\right)$ \

$\left(\begin{array}{lll}1 & 0 & 5 \\ 0 & 1 & -2 \\ 0 & 0 & 0\end{array}\right) \cdot\left(\begin{array}{l}y_{1} \\ y_{2} \\ y_{3}\end{array}\right)=\left\{\begin{array}{c}y_{1}+5 y_{3}=0 \\ y_{2}-2y_{3}=0\end{array} \rightarrow\left\{\begin{array}{c}y_{1}=-5 y_{3} \\ y_{2}=2 y_{3} \\ y_{3} \in R\end{array} \rightarrow y=N(A^T)=\left(\begin{array}{c}-5 \\ 2 \\ 1\end{array}\right) \cdot y_{3}\right.\right.$ \ \

So a basis of $N(A^T)$ is $(-5, 2, 1 )$, and $null(A^T) = dim(N(A^T)) = 1$.

Characteristic of these four subspaces

(1) Orthogonal: Column space and left nullspace are orthogonal, row space and null space are orthogonal. \

$C(A) \perp N\left(A^{\top}\right) \rightarrow(1,2,1) \cdot\left(\begin{array}{c}-5 \\ 2 \\ 1\end{array}\right)=(3,4,7) \cdot\left(\begin{array}{c}-5 \\ 2 \\ 1\end{array}\right)=0$ \

$R(A) \perp N(A) \rightarrow(1,0,3) \cdot\left(\begin{array}{c}-3 \\ 0 \\ 1\end{array}\right)=(0,1,0) \cdot\left(\begin{array}{c}-3 \\ 0 \\ 1\end{array}\right)=0$

(2) Rank-nullity:

Rank + Nullity = Column Number \

dim(Row Space) + dim(Left Nullsapce) = Row Number