Condition for quasi-splitting of special orthogonal group

This becomes clearer when considered in the broader context of special orthogonal groups of non-degenerate quadratic forms over general fields. (You are considering the case of the ground field $\mathbf{R}$, over which an $n$-dimensional non-degenerate quadratic space is characterized up to isomorphism by the signature $(p, q)$: integers $p, q \ge 0$ with $p+q=n$.)

In general, let $(V, Q)$ be a non-degenerate quadratic space of dimension $n \ge 1$ over a field $k$, and one can ask when the connected reductive $k$-group $G = {\rm{SO}}(Q)$ is quasi-split.

By non-degeneracy, if $v \in V - \{0\}$ is a non-trivial zero of $Q$ then there is a hyperbolic plane $H$ in $V$ containing $v$. If $H$ is a hyperbolic plane contained in $V$ (i.e., the span of 2 nonzero $Q$-isotropic vectors $v_0, v'_0$ satisfying $B_Q(v_0,v'_0)=1$, where $B_Q(v,v')=Q(v+v')-Q(v)-Q(v')$ is the associated symmetric bilinear form) then $V$ is the orthogonal sum of $H$ and its $B_Q$-orthogonal space inside $V$.

By iterating this, we arrive at an orthogonal decomposition of quadratic spaces $$(V, Q) \simeq H_1 \perp \dots \perp H_r \perp (V_0, Q_0)$$ where each $H_j$ is a hyperbolic plane (so $0 \le r \le [n/2]$) and $Q_0$ is non-degenerate and $k$-anisotropic (i.e., $Q_0$ has no nontrivial zeros in $V_0$). In other words, in suitable linear coordinates $$Q = x_1 x_2 + \dots + x_{2r-1}x_{2r} + Q_0(x_{2r+1}, \dots, x_n)$$ with $k$-anisotropic $Q_0$. By Witt's Cancellation Theorem, $r$ is the maximal number of pairwise orthogonal hyperbolic planes contained in $V$.

I claim that $r$ is the $k$-rank of $G$; i.e., the common dimension of the maximal $k$-split tori of $G$. More specifically, let $S = \mathbf{G}_m^r \subset G$ be the $k$-split subtorus $\prod {\rm{SO}}(H_j)$. Concretely, the $j$th factor acts through $\mathbf{G}_m$-scaling on an isotropic basis $\{e_j, e'_j\}$ of $H_j$ via $t.e_j = te_j$ and $t.e'_j = t^{-1}e'_j$ and acts trivially on $V_0$ and every $H_i$ for $i \ne j$. Then it is clear from weight-space considerations for the $S$-action on $V$ that $$Z_G(S) = S \times {\rm{SO}}(Q_0).$$ But it is a general (not difficult) fact that a non-degenerate quadratic space over $k$ (with any dimension $\ge 0$, such as $V_0$!) is $k$-anisotropic if and only if the associated special orthogonal group does not contain $\mathbf{G}_m$ as a $k$-subgroup. Hence, the $k$-anisotropicity of $Q_0$ implies that $Z_G(S)/S$ contains no nontrivial split tori, so $S$ is indeed maximal as a $k$-split torus in $G$.

In the structure theory for connected reductive groups over a field $k$, the Levi factors of the minimal parabolic $k$-subgroups are precisely the centralizers of the maximal $k$-split tori, and a parabolic $k$-subgroup is a Borel subgroup precisely when its Levi factors (or better: its quotient by its unipotent radical) is commutative. Hence, $G$ is quasi-split if and only if ${\rm{SO}}(Q_0)$ is commutative, which is to say $\dim V_0 \le 2$.

Returning now to the motivating situation over $\mathbf{R}$, the signature of $(V_0, Q_0)$ is clearly $(r+d_0,r)$ or $(r, r+d_0)$ where $d_0 = \dim V_0$ and the two options for the signature correspond respectively to the $\mathbf{R}$-anisotropic $(V_0, Q_0)$ being positive-definite or negative-definite. Hence, in terms of the classical notation ${\rm{SO}}(p,q)$ clearly $|p-q| = d_0$, so we have proved that $G$ is quasi-split if and only if $|p-q| \le 2$.

Since the OP asked for a reference, I would like to provide some. First of all, a group is quasi-split iff its Satake diagram does not contain black dots. The list of Statake diagrams can be found in many places but most of them are difficult to understand out of context. Of the more useful kind are the tables in

Onishchik, A. L.; Vinberg, Ėrnest Borisovich (1994), Lie groups and Lie algebras III: structure of Lie groups and Lie algebras

For the theory, I found Section 29 of Daniel Bumps book on Lie groups quite useful (see in particular p. 294ff)