How to know if a Feynman diagram is planar?
There is celebrated formula for Euler characteristic $$\boxed{\chi = V - E + F}$$ where $V$ is number of vertices, $E$ - number of edges, $F$ - number of faces involved in the graph. The Euler characteristic is related to the genus $g$ of surface by $\chi = 2 - 2g$. The planar graph corresponds to sphere with $g= 0$.
For the graph (1) you have 2 vertices, 4 edges, 4 faces (exterior face has be taken also into account) - therefore, it is a planar graph. For the graph (2) there seem to be 2 faces, so this graph can be put only on torus, or higher genus surface.
Concerning OP's diagram (1) & (2) and OP's calculations, note that the labelling of the second vertex is reversed, i.e. the color factor becomes
$$ {\rm Tr}(T^a T^b T^c T^d) {\rm Tr}(T^d T^c T^b T^a)~\stackrel{(3.a')+(3.b')}{=}~({\rm Tr}\mathbb{1})^4+\text{subleading terms}, \tag{2.a'}$$
and
$$ {\rm Tr}(T^a T^b T^c T^d) {\rm Tr}(T^d T^c T^{\color{red}{a}} T^{\color{red}{b}})~\stackrel{(3.a')+(3.b')}{=}~({\rm Tr}\mathbb{1})^3 +\text{subleading terms},\tag{2.b'}$$ respectively. We see that diagram (2) has less index contractions [i.e. factors of ${\rm Tr}\mathbb{1}=N$], which is a hallmark of a non-planar diagram.
Here we have repeatedly used the formulas $${\rm Tr}( T^a A) {\rm Tr}( T^a B)~=~ {\rm Tr}(A B)+ \text{subleading terms}, \tag{3.a'}$$
and
$${\rm Tr}( T^a A T^a B) ~=~ {\rm Tr}(A){\rm Tr}(B)+\text{subleading terms}. \tag{3.b'}$$
References:
- D. Tong, Gauge theory lecture notes; chapter 6.