Are groups ordered pairs or sets?

Yes, a group is an ordered pair: the first element of the pair is a set (the underlying function of the group), and the second is a binary function on that set (which, in set theory, is actually a set too).

Saying something like "$G$ is abelian" is an abuse of notation: technically it's incorrect, but it has only one reasonable interpretation (this is only true btw if we aren't considering two different group structures on the same set, which we sometimes do). It's used because it's slightly easier to write than "$(G, \circ)$ is abelian."

Incidentally, given that "$e$" isn't actually part of the tuple $(G, \circ)$, that's also an abuse of notation - one should write $e_G$ (to distinguish it from the identity of some other group) or similar. But, again, we can get away with it in contexts where it won't lead to confusion. Also, it's worth pointing out that many texts treat groups as ordered triples of the form $(G, \circ, e)$.

Often, in maths you encounter structures that are sets plus some structure on that set. For example with groups, you are dealing with a set $G$ and some operation $\cdot: G \times G \to G$ on this set. In topology, you have a set $X$ and a topology $\tau$ on this set. In measure theory you have a set $X$, a $\sigma$-algebra on $X$, denoted $\Gamma$ and a measure on $\Gamma$ denoted $\mu$.

In all of these cases you can describe the thing properly by providing the set and the structure together: a group is $(G,\cdot)$, a topological space is $(X, \tau)$ and a measure space is $(X, \Gamma, \mu)$. I think this is what your notation is about.

This is technically an abuse of language. Conflating a tuple $(A,\ldots)$ defining a set with structure with the set $A$ itself extremely common and I don't remember hearing anyone ever complain about it. This is because the ordered tuple construction is quite artificial and is not the only way to associate a set with the structures we put on it.

For the example of groups, we could instead define a group as an object in the category $\mathbf{Grp}$ of all groups. Now this really is a set. The algebraic structure is then hidden in the morphisms of the category.