Groups of the from $gMg$ in a monoid where $g$ is an idempotent

Yes. You do not even need the finiteness of $M$.

Theorem 1. Let $\left( M,\cdot\right) $ be a monoid with identity $e$. For every idempotent $g\in M$, the set $gMg=\left\{ gxg\mid x\in M\right\} $ is a monoid (with respect to the operation $\cdot$) with neutral element $geg=gg=g$. Let $g$ and $h$ be two idempotents of $M$ such that the monoids $gMg$ and $hMh$ are groups. Then, these groups $gMg$ and $hMh$ are isomorphic.

Let us first show the following simple fact:

Lemma 2. Let $\left( A,\cdot\right) $ and $\left( C,\cdot\right) $ be two groups. Let $\varphi:A\rightarrow C$ be a map. Assume that $\varphi\left( a\right) \varphi\left( b\right) =\varphi\left( ab\right) $ for all $a\in A$ and $b\in A$. Then, $\varphi$ is a group homomorphism from $\left( A,\cdot\right) $ to $\left( C,\cdot\right) $.

Proof of Lemma 2. Let $e$ denote the neutral element of any group. We have assumed that \begin{equation} \varphi\left( a\right) \varphi\left( b\right) =\varphi\left( ab\right) \qquad \text{for all $a\in A$ and $b\in A$.} \label{darij.pf.l2.1} \tag{1} \end{equation} Applying this to $a=e$ and $b=e$, we obtain $\varphi\left( e\right) \varphi\left( e\right) =\varphi\left( \underbrace{ee}_{=e}\right) =\varphi\left( e\right) $. Since $C$ is a group, we can cancel $\varphi\left( e\right) $ from this equality, and obtain $\varphi\left( e\right) =e$. Combined with our assumption \eqref{darij.pf.l2.1}, this shows that $\varphi$ is a monoid homomorphism from $\left( A,\cdot\right) $ to $\left( C,\cdot\right) $. Thus, $\varphi$ is also a group homomorphism (since every monoid homomorphism between groups is a group homomorphism). Lemma 2 is proven. $\blacksquare$

Proof of Theorem 1. The elements $g$ and $h$ are idempotents; thus, we have $gg=g$ and $hh=h$.

Let $p$ be the inverse of the element $ghg$ in the group $gMg$. Then, $p\left( ghg\right) =\left( ghg\right) p=g$ (since $g$ is the neutral element of the group $gMg$).

Let $q$ be the inverse of the element $hgh$ in the group $hMh$. Then, $q\left( hgh\right) =\left( hgh\right) q=h$ (since $h$ is the neutral element of the group $hMh$).

The element $p$ belongs to the group $gMg$, but $g$ is the neutral element of this group. Hence, $pg=gp=p$. Similarly, $qh=hq=q$.

We have \begin{align} gqhghgp=g\underbrace{q\left( hgh\right) }_{=h}gp=ghgp=\left( ghg\right) p=g , \end{align} so that \begin{align} g &= gqhghgp = gqh \underbrace{\left( ghg\right) p}_{=g} = g\underbrace{qh}_{=q}g \label{darij.pf.thm1.2} \tag{2} \\ &= gqg . \label{darij.pf.thm1.3} \tag{3} \end{align} Similarly, \begin{equation} h = hph . \label{darij.pf.thm1.4} \tag{4} \end{equation} We define a map $\alpha:gMg\rightarrow hMh$ by setting \begin{align} \alpha\left( x\right) =hxq \qquad \text{ for every $x\in gMg$.} \end{align} This is well-defined, because every $x\in gMg$ satisfies $hx\underbrace{q} _{\in hMh}\in h\underbrace{xhM}_{\subseteq M}h\subseteq hMh$.

We define a map $\beta:hMh\rightarrow gMg$ by setting \begin{align} \beta\left( y\right) =pyg \qquad \text{ for every $y\in hMh$.} \end{align} This is well-defined, because every $y\in hMh$ satisfies $\underbrace{p}_{\in gMg}yg\in g\underbrace{Mgy}_{\subseteq M}g\subseteq gMg$.

Let us now prove that \begin{align} \alpha\left( a\right) \alpha\left( b\right) =\alpha\left( ab\right) \qquad \text{ for all $a\in gMg$ and $b\in gMg$.} \label{darij.pf.thm1.5} \tag{5} \end{align}

[Proof of \eqref{darij.pf.thm1.5}: Let $a\in gMg$ and $b\in gMg$.

We have $a\in gMg$; thus, there exists some $x\in M$ such that $a=gxg$. Consider this $x$. Thus, $\underbrace{a}_{=gxg}g=gx\underbrace{gg}_{=g}=gxg=a$.

We have $b\in gMg$; thus, there exists some $y\in M$ such that $b=gyg$. Consider this $y$. Thus, $g\underbrace{b}_{=gyg}=\underbrace{gg}_{=g}yg=gyg=b$.

The definition of $\alpha$ yields $\alpha\left( a\right) =h\underbrace{a} _{=ag}q=hagq$ and $\alpha\left( b\right) =h\underbrace{b}_{=gb}q=hgbq$. Multiplying these two equalities, we obtain \begin{align} \alpha\left( a\right) \alpha\left( b\right) =ha\underbrace{gqhg} _{\substack{=g\\\text{(by \eqref{darij.pf.thm1.2})}}}bq=h\underbrace{ag}_{=a}bq=habq . \label{darij.pf.thm1.6} \tag{6} \end{align} On the other hand, the definition of $\alpha$ yields $\alpha\left( ab\right) =habq$. Compared with \eqref{darij.pf.thm1.6}, this yields $\alpha\left( a\right) \alpha\left( b\right) =\alpha\left( ab\right) $. Thus, \eqref{darij.pf.thm1.5} is proven.]

Now, Lemma 2 (applied to $A=gMg$, $C=hMh$ and $\varphi=\alpha$) yields that $\alpha$ is a group homomorphism from $\left( gMg,\cdot\right) $ to $\left( hMh,\cdot\right) $. We shall now focus on proving that $\alpha$ is invertible.

Indeed, let us first show that $\beta\circ\alpha=\operatorname*{id}$. Indeed, let $x\in gMg$. Then, $xg=gx=x$ (since $x$ belongs to the group $gMg$, but $g$ is the neutral element of this group). We have \begin{align} \left( \beta\circ\alpha\right) \left( x\right) &=\beta\left( \underbrace{\alpha\left( x\right) }_{=hxq}\right) =\beta\left( hxq\right) \\ &=\underbrace{p}_{=pg}h\underbrace{x}_{=xg}qg \qquad \text{(by the definition of $\beta$)} \\ &=pgh\underbrace{x}_{=gx}\underbrace{gqg}_{\substack{=g\\\text{(by \eqref{darij.pf.thm1.3})}}}=\underbrace{pghg}_{=p\left( ghg\right) =g}\underbrace{xg} _{=x}=gx=x=\operatorname*{id}\left( x\right) . \end{align} Thus, we have shown that $\left( \beta\circ\alpha\right) \left( x\right) =\operatorname*{id}\left( x\right) $ for every $x\in gMg$. This proves $\beta\circ\alpha=\operatorname*{id}$.

Now, let us show that $\alpha\circ\beta=\operatorname*{id}$. Indeed, let $y\in hMh$. Then, $hy=yh=y$ (since $y$ belongs to the group $hMh$, but $h$ is the neutral element of this group). We have \begin{align} \left( \alpha\circ\beta\right) \left( y\right) &=\alpha\left( \underbrace{\beta\left( y\right) }_{=pyg}\right) =\alpha\left( pyg\right) \\ &= hp\underbrace{y}_{=hy}g\underbrace{q}_{=hq} \qquad \text{(by the definition of $\alpha$)} \\ &=\underbrace{hph}_{\substack{=h\\\text{(by \eqref{darij.pf.thm1.4})}}}\underbrace{y} _{=yh}ghq=\underbrace{hy}_{=y}\underbrace{hghq}_{=\left( hgh\right) q=h}=yh=y=\operatorname*{id}\left( y\right) . \end{align} Thus, we have shown that $\left( \alpha\circ\beta\right) \left( y\right) =\operatorname*{id}\left( y\right) $ for every $y\in hMh$. This proves $\alpha\circ\beta=\operatorname*{id}$.

The maps $\alpha$ and $\beta$ are mutually inverse (since $\alpha\circ \beta=\operatorname*{id}$ and $\beta\circ\alpha=\operatorname*{id}$). Thus, the map $\alpha$ is invertible. Since $\alpha$ is a group homomorphism, this shows that $\alpha$ is a group isomorphism. Thus, there exists a group isomorphism $gMg\rightarrow hMh$ (namely, $\alpha$). This proves Theorem 1. $\blacksquare$

This proof was obtained by lots of experimentation and iterative simplification. Do not ask me for the intuition behind it, for I have none. I suspect it could still be shortened twice, but I am happy enough that I was able to dispose of the finiteness condition and at least some of the ugliest computations. Feel free to improve!