Does $\lim_{h\rightarrow 0}\ [f(x+h)-f(x-h)]=0$ imply that $f$ is continuous?
No. Consider the function $f$ that equal to $0$ on the entire real line, except at $x=0$, where $f(0)=1$. Because $\mathbb{R}-\{0\}$ is open, around any nonzero point, we can find a neighborhood where $f$ is equal to $0$, so the condition clearly holds. And the condition holds when $x=0$, because $x\pm h$ will always be $0$ for $h\neq 0$, and in the definition of limit, we never consider $h=0$.
But $f$ is manifestly discontinuous, so we are done.
For those interested in more about this property, functions satisfying it are called symmetrically continuous functions. It is known that if a function is symmetrically continuous at every point in an interval, then the set of points at which the function is not continuous is small in both the measure sense (has Lebesgue measure zero) and the Baire category sense (is a first category set). However, the non-continuity set of such a function can have cardinality continuum. Indeed, it can have cardinality continuum in every open subinterval of the domain interval.
See Marcin Szyszkowski’s 2000 Ph.D. Dissertation under Chris Ciesielski (at West Virginia University), Symmetrically Continuous Functions, which is freely available on the internet, for much more about these functions.
Incidentally, one way to answer Rudin's question (definitely not what he had in mind) is to observe that there are $c$ many continuous functions and $2^c$ many symmetrically continuous functions (see Miroslav Chlebík's 1991 Proc. AMS paper for this last result).
(added next day) This morning I had a chance to look at Brian S. Thomson's Symmetric Properties of Real Functions (1994). Given a function $f:{\mathbb R} \rightarrow {\mathbb R}$ that is symmetrically continuous at each point, let $D(f)$ be the set of points at which $f$ is not continuous in the usual sense. As of 1994 (and even now, I believe), no exact characterization is known for those subsets of $\mathbb R$ that are equal to $D(f)$ for some symmetrically continuous function $f.$ However, it is known that if $Z$ is a countable union of $N$-sets, then there exists a symmetrically continuous function $f$ such that $Z \subseteq D(f)$ (i.e. $f$ is not continuous at each point in $Z,$ and perhaps also at some points not in $Z$).
A corollary of this is that there exists a symmetrically continuous function $f$ such that $D(f)$ has Hausdorff dimension $1$ in every open interval (i.e. $D(f)$ is "everywhere of Hausdorff dimension $1").$ Note that this is much stronger than $D(f)$ having cardinality $c$ in every open interval, the result I mentioned previously. The "everywhere of Hausdorff dimension $1$" result follows from the fact that there exist, in every open interval, $N$-sets with Hausdorff dimension arbitrarily close to $1,$ and hence each open interval contains a $\sigma$-$N$-set with Hausdorff dimension equal to $1,$ and therefore we can put a $\sigma$-$N$-set of Hausdorff dimension $1$ in each interval of the form $(r,s)$ where $r<s$ are rational numbers (the net result will be a $\sigma$-$N$-set that is everywhere of Hausdorff dimension $1$).
On the other hand, by a result proved by Miroslav Chlebík (not published) that is stated and proved in Thomson's book (pp. 57-59), it follows that no $D(f)$ set can contain the Cantor middle thirds set $C$. In fact, it is the case that if $f$ is symmetrically continuous, then there exists $G \subseteq C$ such that $G$ is residual-in-$C$ and $f$ is continuous at each point of $G.$ That is, $f$ is continuous at "Baire almost all" points of $C.$ Obviously, given my comments about Hausdorff dimension, the Cantor middle thirds set $C$ is not "too big". Instead, the problem has to do with a certain arithmetic-combinatorial property of the distribution of the points in $C.$
Consider the function $$f(x)=\begin{cases}0,&\qquad\hbox{$x$ is irrational}\\1/q,&\qquad\hbox{$x=p/q$ where $\gcd(p,q)=1$ and $q>0$}\end{cases}$$ First, we can see that $f(1+x)=f(x)$, and $f(-x)=f(x)$, so we can restrict $-1<x<1$. Second, I want to show that $\lim_{x\to x_0}f(x)=0$ whenever $-1<x_0<1$. For each $\epsilon>0$, there only exists finity many real numbers $-1<x<1$ such that $|f(x)|\ge\epsilon$, for such $x$ must be rational number with denominator $0<q\le1/\epsilon$. Thus there exists $\delta>0$ such that $|f(x)|<\epsilon$ for all $x$ such that $0<|x-x_0|<\delta$, therefore $\lim_{x\to x_0}f(x)=0$. Hence, $f(x)$ is continuous at irrational number $x=\alpha$ and not at rational number $x=r$, for $f(x)$ is periodic.
$\lim_{h\to0}(f(x+h)-f(x-h))=0$, because $\lim_{x\to r}f(x)=0$ for each rational number $x$, we conclude that $f(x)$ is the counterexample for $f$ is continuous，and the set of discontinuity points is dense on $\Bbb R$.
I don't know whether there's a function $f$ such that $f$ is discontinuous almost everywhere but for all $x_0$ the limit $\lim_{h\to0}(f(x_0+h)-f(x_0-h))=0$.