How could the "Riemann mapping group" be a group?

As pointed out in the comments, $\text{Diff}_+(S^1) / \text{Rot}(S^1)$ is not a group. To see this, note that $\text{SL}(2, \mathbb R)$ is a subgroup of $\text{Diff}_+(S^1)$: The homomorphism $\varphi: \text{SL}(2, \mathbb R) \rightarrow \text{Diff}_+(S^1)$ given by $$\varphi(A)(x,y) = \frac{A(x,y)}{\Vert A(x, y) \Vert}$$ is injective.

Now, the subgroup $\text{Rot}(S^1)$ is actually isomorphic to the subgroup $\text{SO}(2, \mathbb R)$ of $\text{SL}(2, \mathbb R)$ in an obvious way. But $\text{SO}(2, \mathbb R)$ not even normal in $\text{SL}(2, \mathbb R)$ (trial and error will give you a counterexample), so it is also not normal in $\text{Diff}_+(S^1)$. Hence, neither the "Riemann mapping group" nor $\text{Diff}_+(S^1)/\text{Rot}(S^1)$ is a group.