Quick question on an approximation from physics

This is an instance of Taylor's expansion $$ (1+x)^n=1+nx+\frac{n(n-1)}{2}x^2+\dots, $$ with $x=2\varepsilon$ and $n=-1/2$. If you truncate the expansion, the error is an infinitesimal of order higher than the last term left. In your case, the error is of order $\varepsilon^2$, which is small compared to $\varepsilon$.

That's a special case of the Binomial approximation formula

$$(1 + x)^\alpha \approx 1 + \alpha x$$

for $|x|$ small.

This comes from the definition of derivative at $x=0$ of $f(x)=(1+x)^\alpha$: $$(1+x)^{\alpha}=1+\alpha x + o(x)$$ Because $$\left((1+x)^{\alpha}\right)'=\alpha (1+x)^{\alpha -1 }$$