How do fields transform under special conformal transformations?
The fields transform under finite conformal transformations as$^1$ $$ \Phi^a(x') \mapsto {\Phi^a}'(x) = \Omega(x')^\Delta\,D(R(x'))^{\phantom{b}a}_b \,\Phi^b(x')\,. \tag{1}\label{main} $$ as given in equation $(55)$ of $[1]$. Let's break it down:
- $\Delta$ is the conformal dimension of $\Phi$.
- $\Omega$ is the conformal factor of the transformation.
- $D$ is the spin representation of $\Phi$.
- $R$ is the rotation Jacobian of the transformation
So let's compute these things. The spin and the conformal dimension $\Delta$ are given. The first thing we have to look at is the Jacobian. $$ \frac{\partial x^{\prime \mu}}{\partial x^\nu} = \Omega(x') R^\mu_{\phantom{\mu}\nu}(x')\,. $$ This implicitly defines both $\Omega$ and $R$ and it is not ambiguous because we require $R \in \mathrm{SO}(d)$, namely $$ R^{\mu}_{\phantom{\mu}\nu} \,\eta^{\nu\rho}\,R^{\lambda}_{\phantom{\lambda}\rho} \,\eta_{\lambda\kappa}= R^{\mu}_{\phantom{\mu}\kappa}\,. $$ You can immediately see that for the Poincaré subgroup of the conformal group $\Omega(x')= 1$, whereas for dilatations $\Omega(x') = \lambda$ and for special conformal transformations $$ \Omega(x') = \frac{1}{1+2(b\cdot x') + b^2 {x'}^2}\,.\tag{2}\label{omega} $$ This can be proven with a bit of algebra. Using your definition of SCT $$ {x'}^\mu = \frac{x^\mu - b^\mu x^2}{1+-2(b\cdot x) + b^2 {x}^2}\,, $$ one can check $$ \frac{\partial x^{\prime \mu}}{\partial x^\rho} \eta_{\mu\nu} \frac{\partial x^{\prime \mu}}{\partial x^\lambda} = \frac{\eta_{\rho\lambda}}{(1-2(b\cdot x) + b^2 x^2)^2}\,. $$ That means that the Jacobian is an orthogonal matrix up to a factor, which is the square root of whatever multiplies $\eta_{\rho\lambda}$. Then we have to re-express that as a function of $x'$. After some algebra again one finds that it suffices to change the sign to the term linear in $b$.
Finally, how does one compute $R$? Well, it's just the Jacobian divided by $\Omega$. In the case of special conformal transformations one has (there might be mistakes, redo it for safety) $$ R^{\mu}_{\phantom{\mu}\nu} = \delta^\mu_\nu + \frac{2 b_\nu x^\mu - 2 b^\mu (b_\nu x^2+ x_\nu - 2 (b\cdot x) x_\nu) -2b^2 x^\mu x_\nu }{1-2b\cdot x +b^2 x^2}\,, $$ which, as before, needs to be expressed in terms of $x'$.
If you are interested in $\Phi$ scalar then $D(R) = 1$ and you can just plug \eqref{omega} into \eqref{main} to obtain the transformation. If you want to consider also spinning $\Phi$ then it's not much harder.
For spin $\ell=1$ the $D$ is just the identity, namely $$ D(R)^{\phantom{\nu}\mu}_\nu = R^{\phantom{\nu}\mu}_\nu\,. $$ For higher spins one just has to take the product $$ D(R)^{\phantom{\nu_1\cdots \nu_\ell}\mu_1\cdots \mu_\ell}_{\nu_1\cdots \nu_\ell} = R^{\phantom{\nu_1}\mu_1}_{\nu_1}\cdots R^{\phantom{\nu_\ell}\mu_\ell}_{\nu_\ell}\,. $$ Again, by plugging these definitions in \eqref{main} you obtain the desired result.
$\;[1]\;\;$TASI Lectures on the Conformal Bootstrap, David Simmons-Duffin, 1602.07982
$\;{}^1\;\;$The way the transformations are written in the lecture notes linked above differs a bit from Di Francesco Mathieu Sénéchal. The difference is that Di Francesco et al. make an active transformation $x \to x'$ with $$ \Phi(x) \mapsto \Phi'(x') = \mathcal{F}(\Phi(x))\,, $$ while David Simmons Duffin makes essentially the inverse transformation $x' \to x$ $$ \Phi(x') \mapsto \Phi'(x) = \mathcal{F}^{-1}(\Phi(x'))\,. $$ That is why in the above discussion the indices of $R^\mu_{\phantom{\mu}\nu}$ get swapped when passed inside $D$ as $D(R) = R^{\phantom{\nu}\mu}_{\nu}$. And that's also why we get a factor $\lambda^\Delta$ rather than $\lambda^{-\Delta}$ as Di Francesco et al. This is all consistent as long as it is clear what one is doing.