Second derivative "formula derivation"

The only problem is that you’re looking at the wrong three points: you’re looking at $x+2h,x+h$, and $x$, and the version that you want to prove is using $x+h,x$, and $x-h$. Start with $$f\,''(x)=\lim_{h\to 0}\frac{f\,'(x)-f\,'(x-h)}h\;,$$ and you’ll be fine.

To see that this really is equivalent to looking at $$f\,''(x)=\lim_{h\to 0}\frac{f\,'(x+h)-f\,'(x)}h\;,$$ let $k=-h$; then

$$\begin{align*} f\,''(x)&=\lim_{h\to 0}\frac{f\,'(x)-f\,'(x-h)}h\\ &=\lim_{-k\to0}\frac{f\,'(x)-f\,'(x-(-k))}{-k}\\ &=\lim_{k\to 0}\frac{f\,'(x-(-k))-f\,'(x)}k\\ &=\lim_{k\to 0}\frac{f\,'(x+k)-f\,'(x)}k\;, \end{align*}$$

and renaming the dummy variable back to $h$ completes the demonstration.

Using the Taylor series expansions of $f(x+h)$ and $f(x-h)$,

$$ f(x+h) = f(x) + f'(x)h+f''(x)\frac{h^2}{2} + f'''(x)\frac{h^3}{3!}+\cdots $$

$$ f(x-h) = f(x) - f'(x)h+f''(x)\frac{h^2}{2} - f'''(x)\frac{h^3}{3!}+\cdots $$

Adding the above equations gives

$$ \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} = f''(x) + 2\frac{f''''(x)}{4!}h^2+\cdots $$

taking the limit of the above equation as $h$ goes to zero gives the desired result

$$ \Rightarrow f''(x) = \lim_{h\to0} \frac{f(x+h) - 2f(x) + f(x-h)}{h^2} \,.$$

Your formula is correct. You can easily check it by using Taylor (or, more formally, if you only have second derivatives, a second order Mean Value Theorem): $$\begin{multline} \frac1{h^2}\left[f(x+2h)-2f(x+h)+f(x)\right]= \\ \frac1{h^2}\left[ f(x)+2hf'(x)+\frac{4h^2}2f''(x)+o(h^3)-2(f(x)+hf'(x)+\frac{h^2}2f''(x)+o(h^3))+f(x)\right] = \\ \frac1{h^2}\,h^2f''(x) +o(h)=f''(x)+o(h). \end{multline}$$ Your deduction is a little shaky, though, as you are unifying two limits into one without justification. The same argument works, and gives you the formula you wanted, if you start with $$ \frac1{h^2}\left[f(x+h)-2f(x)+f(x-h)\right] $$