Definition of groups
I suppose you could do it.
A common similar reduction is to note that you only need one operator, $g\star h = g\cdot h^{-1}$. Then you can state your axioms in terms of $e=h\star h$, $h^{-1}=e\star h$ and $gh=g\star h^{-1}$. But the axioms become quite noisy, and there are deep reasons we like to talk about associative operations.
So if you have a set $G$ with a binary operation $\star$ with the following properties:
- $G$ is not-empty
- For all $h,g\in G$, $h\star h=g\star g$. From here on, we'll write $e=h\star h$.
- $e\star(e\star h)=h$, $h\star e=h$.
- $h_1\star (h_2\star h_3)=(h_1\star (e\star h_3))\star h_2$.
Once you have such an operation, you can define d $g^{-1}=(g\star g)\star g$ and $g\cdot h=g\star h^{-1}=g\star((h\star h)\star h)$.
One tricky thing is that, without the requirement for an identity, you are going to need to assert that $G$ is non-empty.
There is a deep theoretical reason that we prefer to talk about associative operations first, however. The most fundamental associative operation is function composition. Let $X$ be a set, and let $([X\to X],\circ)$ be the set of all functions from $X$ to itself, with the operation $\circ$ being function composition. $\circ$ is an associative operation.
Turns out, if $(S,\times)$ is any set with an associative operation, then it is equivalent (isomorphic) to some sub-algebra of an $([X\to X],\circ)$ for some set $X$. (You can always use $X=S\sqcup\{I\}$, in fact.) Such "representations" of $(S,\times)$ are a deep common fact in a lot of mathematics, which is related to something called "category theory."
This also indicates why the identity is more primal than inverses.
$([X,X],\circ)$ always has an identity (though $(S,\times)$ might not). So it is easy to "add" an identity to $S$.
However, if $|X|>1$, some elements of $[X,X]$ have no inverses, and if you try to add inverses and keep the associative rule, you end up doing something way more complicated than merely "adding" elements to $S$.
Is it possible to do it the other way around?
Sort of, yes. A semigroup $S$ is a group if and only if for all $a\in S$, $$aS=S=Sa.$$ This is sort of like having inverses first. I suppose it's worth noting that this definition uses neither the identity nor inverses.
However, one might say that to define an inverse in the first place, you need an identity.
Having said that, a non-empty, regular semigroup is a group if and only if it is cancellative: Conditions for a regular semigroup to be a group..
The idea of inverse elements, as such, logically requires the notion of an identity element with respect to which inverses can be defined, but there are partial results in the direction you suggest.
One way to go at this is to define, for a semigroup (associative magma) $S$, a quasi-inverse of an element $a\in S$ to be an element $a^\ast\in S$ for which $aa^{\ast}a = a$. It is then the case that if every element of $S$ has an unique quasi-inverse, then $S$ is a group. We choose an element $a$ in $S$ and show that $aa^\ast$ is, in fact, an identity element for $S$ and $x^\ast$ is an inverse for each $x\in S$. (Uniqueness of quasi-inverses is essential here, as the set of all square matrices over a field has non-unique quasi-inverses under matrix multiplication, but is clearly not a group.)