Non-ellipticity of Yang-Mills equations
Let me try a low tech approach. We start with some definitions from Gang Tian's paper (which means we are in the Euclidean setting). We have $$ A = A_i \mathrm{d}x_i, \qquad A_i:U\to g, $$ where $U\subset\mathbb{R}^n$ is an open set and $g$ is a Lie algebra. Then the Yang-Mills equations for $A$ are $$ \partial_iF_{ij} + [A_i,F_{ij}] = 0, \qquad \qquad (*) $$ where the summation convention is assumed, and $$ 2F_{ij} = \partial_iA_j - \partial_jA_i + [A_i,A_j]. $$ Neglecting the numerical factor $2$, the highest order derivatives appearing in the left hand side of $(*)$ are $$ \partial_i\partial_iA_j - \partial_j\partial_iA_i = \Delta A_j - \partial_j\mathrm{div}A. $$ Note that whatever linearization one choses, this is the principal part of the linearization (because the equation is semilinear). Now the corresponding symbol is $$ p(\xi) = -|\xi|^2 + \xi\otimes\xi. $$ Thinking of $p(\xi)$ as an $n\times n$ matrix, it is clear if $\eta\in\mathbb{R}^n$ is collinear to $\xi$ then $p(\xi)\eta=0$, so $p(\xi)$ is not invertible, hence the associated operator is not elliptic. This explains the basic reason behind non-ellipticity. Let me suggest an exercise: Write out the details of this argument for Riemannian manifolds and for vector bundles. Another good exercise is to compute the principal symbol of the elliptic Einstein equations (or prescribed Ricci curvature problem).
Start with the fact (exercise):
$F_{A+b}=F_A+d_Ab+b\wedge b$.
This is the crux of everything, as you can now linearize the curvature equation. In particular, the YM equation is $F_A^+=0$, so to take its linearization we just take the self-dual part of the above exercise, with $b=ta$, and evaluate $\frac{d}{dt}|_{t=0}$ of it. This should lead you to the desired:
$d_A^+a=0$,
for $a\in\Omega^1(\mathfrak{g}_P)$. Now take it from here, and compute the symbol. In particular, since there is this gauge symmetry group on the space of YM-solutions, to actually get an elliptic system you just add on the linearization of that gauge-action, which ends up being $-d_A^\ast$.