Is $f(a)-f(b) \approx f^{\prime}(b)(a-b)$ for small differences and if so, why?

If $f(x)$ is continuous and differentiable, then, on a small enough scale it can be approximately a line.

In which case $f(x+\delta) \approx f(x) + \delta f'(x)$ for small $\delta$

We could also invoke the mean value theorem.

There exists a $c\in (a,b)$ such that $f'(c) = \frac {f(b) - f(a)}{b-a}$

If $f'$ is assumed to be bounded then $f(a)-f(b)=f'(t) (a-b)$ for some $t \in (a,b)$ so $|f'(t) (a-b)-f'(b) (a-b)|\leq 2M|a-b|$ where $M$ is a bound for $|f'|$.

That is essentially Taylor's theorem:

$\begin{align*} f(x) &= f(a) + f'(a) (x-a) + R_1(x) \end{align*}$

Here $R_1(x)$ is the residue, which for functions with continuous second derivative tends to zero as $x$ tends to $a$ (thus the error is small for $x$ near $a$).