How is the similarity of the structure of two functions defined?

Note that $h$ is a bijection. We say that $f$ and $g$ are conjugate maps. This notion is very useful in linear algebra and dynamical systems for instance, to reduce maps to simpler forms.

You are correct - you can construct two composite functions $h \circ f:A \to B$ and $g \circ h:A \to B$ such that $(h \circ f)(x) = (g \circ h) (x) \quad \forall x \in A$. In other words $h \circ f = g \circ h$.

You could say that functions $f$ and $g$ are isomorphic. In category theory terms, $f, g$ and $h$ form a commutative diagram.