Why does it make sense to talk about the 'set of complex numbers'?

"Element of a group (ring, topological space, etc.)" is simply a common abreviation for "element of the underlying set of a group (ring, topological space, etc.)".

There are algebraic structures like groups, fields, rings, etc., which are tuples of a set and some objects giving the set structure. Usually that's some kind of operation like addition and multiplication, but sometimes special elements of the set are also used. For instance, sometimes groups are defined as a tuple $(G,\ast,e)$ where $G$ is a set, $\ast:G\times G\to G$ a map and $e$ an element of $G$ which together satisfy the group axioms ($\ast$ is associative, for each element of $G$ there exists an inverse, and $e\ast g=g\ast e=g$ for all $g\in G$).

Now when we want to write stuff about the group, writing $(G,\ast,e)$ gets old fast. Which is why essentially everyone just writes down the set instead of the whole tuple, secure in the knowledge that their colleagues will know that they actually mean the tuple when it's clear from context that we're talking about groups. In the same vein, mathematicians will talk about an element of the group when they actually mean an element of the set which is part of the tuple defining the group. But again, all their colleagues are in the know, so it's ok.

The same goes for the complex numbers. Yes, technically, the complex numbers are the tuple $(\mathbb R^2,+,\cdot)$, so an element of "the complex numbers" is not an element of the set $\mathbb C:=\mathbb R^2$. But we talk about elements of the complex numbers anyway because it would be tiresome to talk about elements of the underlying set of the complex numbers. Everyone knows what you mean anyway.

TL;DR: Mathematicians are lazy, so they talk about elements of a group even if it's not technically correct.

In any case, there is a trivial isomorphism between $(\mathbb R^2,+,\times)$ where the multiplication is that of the complex, and $(\mathbb C,+,\times)$. From an abstract point of view, these structures are interchangeable.

I don't see any reason to avoid the definition of a set denoted as $\mathbb C$, where the elements are equivalently written like

$$z:=(a,b)$$ or $$z:=a+ib$$ where $a,b\in\mathbb R$. Whether it is considered a different set from $\mathbb R^2$ or not seems a useless/irrelevant question. Anyway, if they are considered different, allowing to mix elements of $\mathbb C$ and $\mathbb R^2$ (e.g. defining addition between them) would seem a paranoid idea.