Are integral curves of a vector field $X$ such that $\nabla_X X = 0$ geodesics?

$X(X,X)=0$ so that $|X|$ is constant on an integral curve $c(t)$. Since $c$ is a constant speed, so by an assumption, $c$ is a geodesic.

The answer is yes. I believe we can prove it with the pullback connection. On any curve $\alpha$ we have the induced pullback connection $\alpha^\ast \nabla$. The geodesic equation becomes with it $$\alpha^\ast \nabla_{d/ds} \alpha' = 0$$

We show this is satisfied. Let $\gamma$ be integral curve. Thus we have that $X \circ \gamma = \gamma'$. It turns out that this means that $\gamma' = \gamma^\ast X$, so that it is a pullback section. Thus $$\gamma^\ast \nabla_{d/ds} \gamma' =\gamma^\ast \nabla_{d/ds} \gamma^\ast X, $$

and by definition of the pullback connection $$\gamma^\ast \nabla_{d/ds} \gamma' =\gamma^\ast \nabla_{\gamma_\ast d/ds} X, $$

but $\gamma_\ast d/ds = X \circ \gamma$ and since the connection is $C^\infty$ linear on the below slot we have $$\gamma^\ast \nabla_{d/ds} \gamma' =\gamma^\ast \nabla_{X} X, $$

so when your condition is satisfied the curve is a geodesic.