Cech to derived spectral sequence and sheafification

Yes, this is true in general.

It suffices to show the stalks vanish. Pick $x \in X$ and take an injective resolution $0 \to {\cal F} \to I^0 \to \cdots$. For any open $U$ containing $x$, we get a chain complex

$$0 \to I^0(U) \to I^1(U) \to \cdots$$

whose cohomology groups are $H^p(U,{\cal F}|_U)$.

Taking direct limits of these sections gives the chain complex

$$0 \to I^0_x \to I^1_x \to \cdots$$

of stalks, which has zero cohomology in positive degrees because the original complex was a resolution. However, direct limits are exact and so we find

$$0 = {\rm colim}_{x \in U} H^p(U,{\cal F}|_U) = {\underline H}^p({\cal F})_x$$

as desired.

Generally, cohomology tells you the obstructions to patching local solutions into global solutions, and this says that locally those obstructions vanish.

Here's another short proof: denote by $I,J$ the inclusion of sheaves on X into presheaves and sheafification respectively, then

$$ \underline{H}^p(\mathcal{F})\cong R^pI(\mathcal{F}). $$

Since $J$ is exact,

$$ J\circ R^pI\cong R^p(J\circ I) $$

and the later vanishes for $p>0$ as $J\circ I=id_{\mathfrak{Ab}(X)}$. So $\underline{H}^p(\mathcal{F})$ sheafifies to 0 for $p>0$.

I do not think that sheaves of abelian groups need to be locally acyclic. Let me say what I mean in an example. Take $X=\mathbb{C}^{2}$ with the classical (metric) topology. Let $\mathcal{F} = \mathbb{Z}_{D}$ where $D=\mathbb{C}^{\times} \times \{0\}$.

Then for any arbitrarily small polydisk $U$ containing $(0,0)$ we have $H^{1}(U,\mathbb{Z}_{D}) = H^{1}(U \cap D, \mathbb{Z})$

is not vanishing. It seems in this example that $\underline{H}^{1}(\mathcal{F})_{(0,0)}$ does not vanish.