are non-degenerate critical points always isolated?

Yes, this is a right way of thinking about the question. You can see this in several ways.

1: The Morse lemma: The Morse lemma declares that about any nondegenerate critical point $p$ of a smooth function $f$ there is a coordinate neighborhood $x_i$ so that in those coordinates, $f(x_i) = x_1^2 + \cdots + x_i^2 - x_{i+1}^2 - \cdots - x_n^2 + f(p)$. It's an easy exercise to compute the gradient in these coordinates, and you can immediately see that the only zero of the gradient in the neighborhood is at $p$. (In this case, $p = 0$ and the claim is that you can precompose $f$ with a diffeomorphism $\varphi$ of $\mathbb{R}^n$ so that $f\circ\varphi = \sum x_i^2 - \sum x_j^2$.)

2: Directly: Take a curve (easiest, a straight line) $\gamma$ through $0$, and for simplicity's sake, parametrize the curve so that $\gamma(0)=0$. Consider $f|_\gamma$. Then $f'(t) = (\nabla f\cdot \gamma')(t)$ and $f''(t) = (\gamma'^THess_f\gamma')(t)$. Leaving out some details you should verify as an exercise, the Hessian is invertible, so in a neighborhood of $0$, $f$ has a local extremum, hence the zero of $f'$ is isolated. This is true for any curve, thus, every direction, so $0$ is an isolated critical point. This formalizes your intuition about "every direction."