Possible isometry groups of open manifolds

Let $M$ be the triply punctured 2-sphere (i.e. $M=S^2-\{p_1, p_2, p_3\})$); one can also take any noncompact connected surface of finite topological type as long as $\chi(M)<0$ but the proof is a bit more involved.

Suppose that $g$ is a Riemannian metric on $M$. To simplify matters, I consider only orientation-preserving isometries.

Then $\mathrm{Isom}(M,g)< \mathrm{Conf}(M,g)$. I claim that $\mathrm{Conf}(M,g)$, the group of conformal automorphisms of $(M,g)$, is finite. The mapping class group of $M$ (the group of self-diffeomorphisms of $M$ modulo isotopy) is finite (isomorphic to the dihedral group of order $6$); hence, it suffices to prove that if $f: (M,g)\to (M,g)$ is a conformal automorphism isotopic to the identity then $f=\mathrm{id}$. This is quite standard (most likely, you will find it in Farb-Margalit's book on the mapping class group).

Note that $f$ is isotopic to the identity if and only if it induces an inner automorphism of $\pi_1(M)$.

By the uniformization theorem, $(M,g)$ is conformal to the quotient of the hyperbolic plane $H^2$ by a discrete group of isometries $\Gamma$ (isomorphic to $F_2$, the free group on two generators). The map $f$ then lifts to an isometry $F$ of $H^2$ (a linear-fractional transformation in, say, the Poincare disk model of $H^2$) which commutes with every element of $\Gamma$ (since $f$ induces an inner automorphism of $\pi_1(M)\cong \Gamma$). In particular, $F$ acts on the boundary circle $S^1$ of $H^2$ fixing fixed points of all elements of $\Gamma$. Since $\Gamma$ is free of rank 2, the set of fixed points of its nontrivial elements is infinite. Hence, $F$ fixes at least three points of $S^1$. Hence (being a linear-fractional transformation), $F=\mathrm{id}$; thus, $f=\mathrm{id}$.

To conclude, every Riemannian metric on $M$ has finite group of isometries.

Edit: I can prove the same claim for each noncompact complete hyperbolic $n$-manifold of finite volume, but a proof is more difficult.

In general, the question can be reformulated in purely topological terms: Which smooth noncompact connected manifolds admit smooth proper actions of noncompact Lie groups? I suspect, this was studied in 1960s...

The answer is sometimes yes:

If $M$ admits a complete vector field $X$, such that its flow $\operatorname{Fl}^X: \mathbb R\times M \to M$ is a proper action, then there exists a Riemannian metric which is invariant under the flow, by the following

Theorem. If M is a proper G-manifold, then there is a G-invariant Riemann metric on M.

which is due to Palais if I remember correctly. A proof of this theorem can be found in 6.30 of Topics in Differential Geometry. AMS, 2008.

Finally, the isometry group then contains a non-compact 1-parameter group (this is not enough: it might be dense) which moves each point towards infinity for $t\to\infty$. So the isometry group cannot be compact (in the compact-open topology).

The existence of such a vector field is a nontrivial condition, since then $M$ is the total space of real line bundle, as pointed out by Misha Kapovich. This is actually proved in 29.21 of 1.

See his answer for an example where the answer is no.

ADDED:

The converse is also true: If the (connected component of the) isometry group is not compact, it contains a closed 1-parameter group whose generating vector field then has proper flow.