What is a Hamiltonian of a System?

Quoting Classical Mechanics (3rd ed.), Goldstein:

In a very literal sense, the Hamiltonian is the generator of the system motion with time.

The motion of the system in a time interval $\text dt$ can be described by an infinitesimal contact transformation generated by the Hamiltonian.

I will go for a mathematical answer. Hopefully, you will get something from it. Sorry if I will be a little bit synthetic.

Take the configuration space $M$ (with charts $q_u : U \to \mathbb{R}^n , U\subset M$) of your physical system, which we suppose is a differentiable manifold, and consider its cotangent bundle $T^*M$ (with charts $(q_u,p_u))$. Then you can define a natural 2-form $\omega := \sum_idq_i\wedge dp_i$ which turns out to be non-degenerate and closed.

Then you might think: since this $\omega$ emerges quite naturally, it would be nice if time evolution didn't change it. So you ask yourself: what kind of vector fields $v$ generate flows $\Phi_t^v$ that preserve $\omega$ (i.e. $\Phi_t^{v\ *}\ \omega = \omega\ \ \forall t$, with $^*$ meaning pull-back)?

The answer is given by $L_v\omega = 0$, with $L$ being the Lie derivative. Applying Cartan's magic formula for Lie derivatives of differential forms, you get the condition $0 = d(\iota_v\omega) + \iota_v(d\omega) = d(\iota_v \omega)$ since $\omega$ is closed.

Then, at least locally (depending on the topology of your system), this means that there is a function $H: T^*M \to \mathbb{R}$ such that $\iota_v\omega = dH$. This $H$ is what (in full generality) you can call Hamiltonian.

Since $\omega$ is non-degenerate, we can invert the relation and write $v = P(dH)$ (P is called "Poisson Tensor") and conclude that there is a natural flow ("Hamiltonian Flow") on the cotangent bundle of the configuration space associated to any function $H$. It so happens that if you are doing Newtonian mechanics and let $H$ be the energy, then the flow you get is the time evolution of the system. But the formalism is more general than that.

For practical purposes, the Hamiltonian formulation does express conservation of energy and momentum in generalised coordinates. It is possible to construct counter-examples, using time varying coordinates. Time varying coordinates mess with the definitions of energy and momentum, but this is artificial. Coordinates are a human choice, and the sensible choice is usually to make things conceptually simple. Indeed the motivation for both the Lagrangian and Hamiltonian formulations was that celestial motions are best analysed in polar coordinates, but the Newtonian formulation implicitly used Cartesian coordinates.

We are not now so constrained. When Hamilton created his formulation vectors had barely been thought of and the vector methods we now have were not available. Also, Hamilton was working at about the same time as Coriolis showed the vital role of conservation of energy in Newtonian dynamics. Consequently, Hamilton derived his formulation from the Lagrangian formulation, which was an earlier method for using generalised coordinates in the absence of the maths we now have for vectors. Modern treatments of the Newtonian formulation express it using vectors from the outset, leading to much better ways of treating generalised coordinates.

Although the original methods of Lagrange and Hamilton are still commonly taught, they are actually of more interest to history than to physics. Indeed, Prof Goddard, who taught me mechanics at Cambridge said exactly that. They should be obsolete if it were not for the role of the Hamiltonian in quantum mechanics and the method of canonical quantisation by which quantum mechanics is often introduced. In quantum mechanics the Hamiltonian operator, which determines time evolution, corresponds precisely to energy. If one is interested in

deep physical implications about the nature of how things work

the answers should be found in quantum mechanics, not in reformulations of classical mechanics.