A semigroup $X$ is a group iff for every $g\in X$, $\exists! x\in X$ such that $gxg = g$

I believe this is well-known to those working in regular/inverse semigroups. So well-known in fact that one can even find proofs on the web, e.g. see this proof on PlanetMath. Alas, I cannot recall the history of this result, though I vaguely recall reading something about such decades ago.

For completeness, I reproduce below the proof presented on PlanetMath (slightly edited).

Theorem $\ $ A non-empty semigroup $S$ is a group if and only if for every $x\in S$ there is a unique $y\in S$ such that $xyx=x$.

Proof $\ $ Suppose that $S$ is a non-empty semigroup, and for every $x\in S$ there is a unique $y\in S$ such that $xyx=x.$ For each $x\in S,$ let $x'$ denote the unique element of $S$ such that $\,xx'x=x.\ $ Note $\,x(\color{blue}{x'xx'})x=(xx'x)x'x=x\color{#C00}{x'}x=x,\,$ so, by uniqueness, $\color{blue}{x'xx'}=\color{#C00}{x}',$ and therefore $\color{blue}x = \color{#C00}{x}''.$

For any $x\in S,$ the element $xx'$ is idempotent, by $(xx')^2=(xx'x)x'=xx'.$ As $S$ is nonempty, we infer that $S$ has at least one idempotent. If $i\in S$ is idempotent, then $ix =ix\color{#0A0}{(ix)'}ix=ix\color{#C00}{(ix)'i}ix,$ so, by uniqueness, $\color{#C00}{(ix)'i}=\color{#0A0}{(ix)'},$ hence $(ix)'=(ix)'(ix)''(ix)'=\color{#C00}{(ix)'i}x(ix)'=\color{#0A0}{(ix)'}x(ix)',$ so, by uniqueness, $x = (ix)''=ix.$ So every idempotent $i$ is a left identity, and, by a symmetry, a right identity. Therefore, $S$ has at most one idempotent element. Combined with the previous result, this means that $S$ has exactly one idempotent element, denoted $e$ . We have shown that $e$ is an identity, and that $xx'=e$ for each $x\in S,$ hence $S$ is a group.

Conversely, if $S$ is a group then $xyx=x$ clearly has a unique solution, namely $y=x^{-1} . $ $\ $ QED

I am just putting some sources here. Bill Dubuque already did all the work:

This book may be of some interest: Mark V. Lawson, Inverse Semigroups: The Theory of Partial Symmetries, at Google Books

I came across this book in this Wikipedia article, which gives some of the history of Inverse Semigroups.

Also, there is this shorter PDF by the author of the book whose link I gave above, and what you state is probably Proposition 2.4 in the notes (although it is stated in a different way) after some other things have already been proved.