In what sense of "structure" do group homomorphisms "preserve structure"?

The "structure" preserved is how the elements relate to each other under the operations of the algebraic structure. For example, for commutative rings, if $\ a = b^2 - c^2 = (b-c)(b+c)\ $ is a difference of squares then applying a ring hom $\,h\, :\, r\mapsto \bar r\,$ shows that it remains a difference of squares in the image ring, viz. $\ \bar a = \bar b^2-\bar c^2 = (\bar b - \bar c)(\bar b +\bar c),\,$ so this particular ring-theoretic structure is preserved. Similarly preserved are polynomial expressions, i.e. expressions composed of basic ring operations (addition, multiplication) and constants $\,0,1.\,$ For example, a root of a polynomial remains a root of the image of the polynomial. So we can prove that a polynomial has no roots by showing it has no roots in a simpler homomorphic image, e.g. a ring with size so small that we can easily test all of its elements to see if they are roots, e.g. common parity arguments: $\,f(x) = x(x\!+\!1)+2n\!+\!1\,$ has no integer roots since $\,x(x\!+\!1)\,$ is even, so adding $2n\!+\!1$ yields an odd hence $\ne 0;\,$ equivalently $\!\bmod 2\!:\ f(0)\equiv 1\equiv f(1),\,$ i.e. $\,f\,$ has no roots mod $\,2,\,$ hence it has no integer roots. This idea generalizes as follows

Parity Root Test $\ $ A polynomial $\,f(x)\,$ with integer coefficients has no integer roots when its $\rm\,\color{#0a0}{constant\,\ coefficient}\,$ and $\,\rm\color{#c00}{coefficient\,\ sum}\,$ are both odd.

Proof $\ $ If so then $\ \color{#0a0}{f(0)} \equiv 1\equiv \color{#c00}{f(1)}\,\pmod 2,\ $ i.e. $\:f\:$ has no roots in $\,\Bbb Z/2 = $ integers mod $\,2,\,$ therefore $\,f\,$ has no integer roots. $\ $ QED

In the same way, we can often reduce problems to "smaller", simpler problems in modular images. Because homs preserve the ambient algebraic structure, as above, we can often deduce information about the original problems from information gleaned in the simpler modular images. Such problem solving by modular reduction is an algebraic way of "dividing and conquering".

Let's let $f:G\rightarrow H$ be a group homomorphism. We might say that $f$ preserves the structures of equalities (without quantifiers) - that is, we can say that, if, in $G$ $$xy=z$$ then we can embed this system into $H$ by setting $$f(x)f(y)=f(z).$$ More generally, we could embed any set of equalities into $H$ similarly - for instance, if we have that two specific elements commute, then we can find two (not necessarily distinct) elements in $H$ which commute - in this sense, we find the structure of $G$ in $H$ by using $f$, even if the structure shows up in some trivial way (i.e. the identity element satisfies every system of equation writable with group operations, hence the trivial homomorphism is a real thing). That is, we consider "structure" to mean "sets of equalities involving group operations"

Notably, the structure that a homomorphism does not preserve but an isomorphism does includes inequality and anything to do with quantifiers. For instance, if in $G$ we have $$\forall x,y [xy=yx]$$ then an isomorphism lets us translate this same statement over to $H$ - quantifier and all, meaning $H$ is abelian. It also lets us embed statements involving inequality, like how the system $$x^2=e$$ $$x\neq e$$ might be solvable in $\mathbb Z/2\mathbb Z$, but not in quotient groups (i.e. images of homomorphisms), like the identity group - so the additional structure of inverses is required here. Now, we can't use the above to define an isomorphism vs. a homomorphism, but it does serve as an example of how homomorphisms preserve some very specific structure, limited to using only the group operations and equality, where isomorphisms give the stronger conditions necessary to make more general manipulations equivalent in both groups.

It preserves the structure in the sense that $f(e_G)=e_H$ and $f(a\cdot_Gb)=f(a)\cdot_Hf(b)$, assuming that $f\colon G\to H$ is a homomorphism of course.

This is a weak preservation, but it preserves some structure nonetheless. Consider, for example, any bijection between $\Bbb Z$ and $\Bbb Q$. Does it preserve any additive structure? It does not.

Saying that two groups "have the same structure" means that they are isomorphic; but just having a map which preserve, or rather does not destroy, the group structure is often sufficiently interesting.