Is the symmetric definition of the derivative equivalent?
As I noted in a comment to the other answer, Milly's computation is incorrect. I am posting this answer to rectify the situation. The symmetric derivative is defined to be \begin{align} \lim_{h\to 0} \frac{f(x+h)-f(x-h)}{2h}. \end{align} If $f$ happens to be differentiable, then the symmetric derivative reduces to the usual derivative: \begin{align} \lim_{h\to 0} \frac{f(x+h)-f(x-h)}{2h} &= \lim_{h\to 0} \frac{f(x+h)-f(x)+f(x)-f(x-h)}{2h} && (\text{add zero}) \\ &= \frac{1}{2} \left( \lim_{h\to 0} \frac{f(x+h)-f(x)}{h} + \lim_{h\to 0} \frac{f(x) - f(x-h)}{h}\right) \\ &= \frac{1}{2} \left( \lim_{h\to 0} \frac{f(x+h)-f(x)}{h} - \lim_{-h\to 0} \frac{f(x+h) - f(x)}{-h}\right) && (\ast)\\ &= \frac{1}{2} \left( f'(x) + f'(x) \right) && (\text{since $f'(x)$ exists}) \\ &= f'(x). \end{align} I'm begin a little trixy at $(\ast)$. Notice that we can think about the difference quotient as being the slope of a secant line through the points $(x,f(x))$ and $(x+h,f(x+h))$. This slope is given by \begin{equation*} \frac{f(x+h)-f(x)}{(x+h)-x}. \end{equation*} Multiplying through by $-1$, this becomes \begin{equation*} -\frac{f(x+h)-f(x)}{(x+h)-x} = \frac{f(x+h)-f(x)}{x-(x+h)} = \frac{f(x+h)-f(x)}{-h} \end{equation*} Taking $h$ to zero (which is the same as taking $-h$ to zero) on the left gives $-f'(x)$, justifying $(\ast)$. Again, this proves the key statement:
Proposition: If $f$ is differentiable at $x$ (in the usual sense), then $f$ is symmetric differentiable at $f$, and the symmetric derivative agrees with the usual derivative.
The converse does not hold. The usual example is the absolute value function which is not differentiable at zero, but which is symmetric differentiable at zero (with derivative zero): \begin{equation*} \lim_{h\to 0} \frac{|0+h|-|0-h|}{2h} = \lim_{h\to 0} \frac{|h|-|h|}{2h} = \lim_{h\to 0} \frac{0}{2h} = 0. \end{equation*} In particular, this demonstrates that the symmetric derivative is a generalization of the usual derivative. It cannot be equivalent, because it can meaningfully define the derivative of a larger class of functions.
As to the why? of the symmetric derivative (and why we don't use it instead of the usual derivative), I think that adequate answers can be found attached to this question and in comments elsewhere on MSE.