Groups where no elements commute except for the trivial cases

Let us consider a finite group $G$ with this property.

Let $P \ne 1$ be a Sylow $p$-subgroup of $G$. If $g$ is an element of order $p$ in $Z(P)$, then every element of $P$ commutes with $g$, so that $P = \langle g \rangle$.

Thus all Sylow subgroup have prime order, that is, the order of $G$ is squarefree.

It follows that $G$ is metacyclic, and actually the semidirect product of two cyclic groups (I am thinking Schur-Zassenhaus or Hall's theorems, but it might be simpler than that), which by an argument similar to the one above have to be of prime order.

It follows that the finite groups with this property are the non-trivial semidirect products of a cyclic group of prime order $p$ by a cyclic group of prime order $q \mid p - 1$.

PS This related discussion may be of interest.

To provide contrast to Andreas Caranti's answer, a Tarski monster $p$-group is an example of an infinite group with these properties.

If $G$ is a Tarski monster $p$-group then (by definition) every proper nontrivial subgroup is cyclic of order $p$. The centralizer of any nontrivial element $g$ is a proper subgroup (since $G$ has trivial center), and thus must be $\langle g\rangle$.

Since you haven't limited your question to finite groups, another example is the free group on $n$ generators, where $n \gt 1$.

Edited to add: As noted below in the comments, this isn't correct because $g=x^2, h=x$ is a counterexample (where $x$ is a generator of the group).