Example of compact Riemannian manifold with only one closed geodesic.

First of all, you have to exclude constant maps $S^1\to M$ from consideration: They are all closed geodesics. Secondly, you have to talk about geometrically distinct closed geodesics: Geodesics which have the same image are regarded as "the same". Then, it is a notorious conjecture/open problem:

Conjecture. Every compact Riemannian manifold of dimension $n >1$ contains infinitely many geometrically distinct nonconstant geodesics.

See for instance this survey article by Burns and Matveev.

This is known for surfaces (with the only hard case when the surface is diffeomorphic to $S^2$ in which case the result is due to Bangert and Franks) and for many higher-dimensional manifolds. However, the problem is open already when $M$ is diffeomorphic to the sphere $S^n$, $n\ge 3$.

Edit. I recently found a preprint which claims to prove the conjecture on closed geodesics:

S. Charles, The Existence of Infinitely Many Geometrically Distinct Non-constant Prime Closed Geodesics on Riemannian Manifolds, 2018.

The paper is still unpublished and I do not know if the proof is correct.

If you analyze the geodesics using Clairaut's relation, you'll find that the only closed geodesic on a hyperboloid of one sheet is the central circle. Indeed, the same holds for a concave surface of revolution of the same "shape" as the hyperboloid of one sheet.

EDIT: Apologies for missing the crucial compactness hypothesis.