The Norton's dome suggesting non-determinism in classical physics

The function is a valid solution of the differential equation. There is nothing wrong mathematically.

Whether it follows that classical mechanics is not deterministic depends on what you consider to be classical mechanics. Note that the equation of motion was derived using a constraint (that the trajectory lies on the surface of the dome), which is an idealization of the forces actually acting between the particle and the dome. If the dome itself were modelled as a classical body of finite mass, the particle could not suddenly gather momentum out of nowhere in violation of the law of conservation of momentum. Thus, for one, you have to consider idealized constraints as part of classical mechanics to reach your conclusion based on this example.

For other reflections on and possible objections to the argument from Norton's dome, see e.g. Malament, D.B., Norton's Slippery Slope, Philosophy of Science, $75$ (December $2008$), pp. $799$-$816$, and for other arguments in favour of indeterminism in classical mechanics, see e.g. the article on causal determinism in the Stanford Encyclopedia of Philosophy.