Proving existence of $T$-invariant subspace

Hint:

Consider the minimal polynomial $\mu_T$ of $T$. If it has degree $1$ or $2$, what do you know about $T$?

If it has degree $3$, factor it into $\mu_T = pq$. What do you know about $p(T)$?

Kernels, eigenspaces and complements may help you.


Edit: Also, I'm curious where this question comes from. Have you considered the example $$T = \begin{pmatrix} 1 & 1 & 0 \\ 0 & 1 & 1 \\ 0 & 0 & 1 \end{pmatrix}?$$ Note that this is already in Jordan normal form. Now choose $v := e_2$ as the second standard basis vector. The image $Tv$ lies in $W := \langle e_1, e_2 \rangle$ and $e_1$ is an eigenvector. So $W$ is $T$-invariant of dimension $2$.

This works similarly in general. If the minimal polynomial splits (into linear factors), the Jordan normal form gives you the answer: Some generalized eigenspace, or a subspace thereof (or, in the diagonalisable case, a union of eigenspaces).

The only other thing that can happen is that the minimal polynomial contains an irreducible factor $f$ of degree $2$. In this case, consider $\operatorname{ker} f(T)$.


Edit2: More details.

When you have the Jordan form, you can read off several invariant subspaces. First of all, eigenspaces are obviously invariant. If you have more than one eigenspace or an eigenspace of dimension $> 1$, just choose the span of suitable eigenvectors as your $T$-invariant space.

Now, if you have a Jordan block of higher dimension, you observe how $T$ acts on the corresponding basis vectors: in my example, $e_3$ is mapped into $\langle e_2, e_3 \rangle$, $e_2$ is mapped into $\langle e_1, e_2 \rangle$. Thus, $\langle e_1 \rangle$, $\langle e_1, e_2\rangle$ and $\langle e_1, e_2, e_3 \rangle$ are all $T$-invariant. So $\langle e_1, e_2 \rangle$ is the $2$-dimensional $T$-invariant space you're looking for.

This covers the cases when the characteristic polynomial splits. If it doesn't (over $\mathbb{R}$), you need to consider the kernel $\operatorname{ker} f(T)$ where $f$ denotes the irreducible factor of degree $2$, as I wrote above. Choose, for example, $$T = \begin{pmatrix} -1 & -2 & -2 \\ 1 & 1 & 0 \\ 0 & 0 & 1 \end{pmatrix}$$ and see what happens.


Edit3: Expanding on my (hopefully) last comment to this answer:

The minimal polynomial of the example above is $(x-1)(x^2+1)$. Moreover, we have $$\operatorname{ker} ( T^2 + 1 ) = \langle e_1, e_2 \rangle \text{ and } \operatorname{ker} ( T - 1 ) = \langle e_2 - e_3 \rangle.$$ Choose any vector in the first space, say $v_1 := e_1$. Then choose $v_2 := Tv_1 = -e_1+e_2$. We find $Tv_2 = -e_1$. Thus, choosing the basis $B := (v_1,v_2,e_2-e_3)$, we obtain $$ {}^BT^B = \begin{pmatrix} 0 & -1 & 0 \\ 1 & 0 & 0 \\ 0 & 0 & 1 \end{pmatrix}$$ and $\langle v_1, v_2 \rangle$ is clearly a $T$-invariant subspace of dimension $2$.

Note also that clearly $K := \operatorname{ker}(p(T))$ is a $T$-invariant subspace for any polynomial $p$:

If $x \in K$, then $p(T) Tx = Tp(T)x = 0$, so $Tx \in K$. We just needed to find a polynomial $p$ such that $\operatorname{ker}(p(T))$ has dimension $2$. (We do not always find such a polynomial. We do, however, always find suitable $T$-invariant subspaces of the kernel.)


Let $F$ be any field, $V$ a finite-dimensional $F$-vector space, and $T: V \rightarrow V$ an $F$-linear map.

For $v \in V$, there is a unique monic polynomial $P_v(t)$ of least degree such that $P_v(T)v = 0$: this is the local minimal polynomial at $v$. The degree of $P_v$ is the least natural number $d$ such that $v, Tv, T^2v,\ldots,T^d v$ are linearly dependent. Therefore it is equal to the dimension of $[v] = \operatorname{span} \{T_i v\}_{i=0}^{\infty}$: notice that $[v]$ is the minimal $T$-invariant subspace of $V$ containing $v$.

Thus a reasonable strategy for showing that $T$ has a $2$-dimensional invariant subspace is to try to show that there is a vector $v \in V$ such that $P_v$ has degree $2$.

Now here is a helpful fact: the set of all local minimal polynomials $P_v$ of vectors $v$ is precisely the set of monic divisors of the ("global") minimal polynomial $P$ of $v$. To see this, first one shows that $P = P_v$ for some $v$ ("Local Attainment Theorem"), and then one checks that for any $v \in V$ and any monic polynomial $f$ dividing $P_v$, $P_{f(T)v} = \frac{P_v}{f}$: for proofs, see Lemma 1.15 and Theorem 1.16 of these notes.

Now I claim the following result, which is stronger than what the OP asked for.

If $F = \mathbb{R}$ and $\operatorname{dim} V \geq 2$, then $V$ has a $2$-dimensional $T$-invariant subspace.

Proof: Let $P$ be the minimal polynomial of $T$.

Case 1: Suppose $P$ has degree $1$. Then $P = (t-\alpha)$ for some $\alpha \in \mathbb{R}$ and $T$ is just the scalar endomorphism $v \mapsto \alpha v$. Then every subspace is $T$-invariant, so because $\dim V \geq 2$, there is a $2$-dimensional invariant subspace.

Casae 2: Suppose $P$ has degree at least $2$. By the Fundamental Theorem of Algebra, the irreducible factors of any real polynomial all have degree either $1$ or $2$. If there is a degree $2$ irreducible factor $f$, then as above there is a vector $v$ such that $P_v = f$ and thus $[v]$ is a $2$-dimensional $T$-invariant subspace. Otherwise $P$ is a product of linear factors, and since it has degree at least $2$, again it has a monic divisor $f$ of degree $2$ and thus again a $2$-dimensional $T$-invariant subspace $[v]$.

Final Remark: The proof of the displayed fact above uses only the following property of $\mathbb{R}$: the only possible degrees of irreducible polynomials $f \in \mathbb{R}[t]$ are $1$ and $2$. The fields $F$ with this property are precisely the algebraically closed fields (the only possible degree is $1$) and the real-closed fields, characterized, for instance, by: $F$ is not algebraically closed but $F[t]/(t^2+1)$ is).

Post-Final Remark: The "Local Attainment Theorem" is proved on p. 10 of my notes. The proof is not so difficult but uses a little more algebra of polynomials than might be common in a linear algebra course. So I wanted to mention an alternate approach which works over any infinite field ($\mathbb{R}$ is infinite). Namely, for any proper monic divisor $f$ of the global minimal polynomial, the (invariant) subspace $V_f$ of all vectors $v$ such that $f(T)v = 0$ must be proper: if it were all of $V$ then we've found a smaller degree polynomial such that $f(T) = 0$. There are only finitely many monic divisors of $P$. Since a vector space over an infinite field cannot be a finite union of proper subspaces -- a fact which is not hard to prove in general but is almost obvious over $\mathbb{R}$ -- we see that in fact "most" vectors have local minimal polynomial equal to $P$.