Time as a Hermitian operator in quantum mechanics
Time is not a variable in Quantum Mechanics (QM), it's a parameter — much in the same way as it is in Classical (Newtonian) Mechanics.
So, if you have a Hamiltonian, e.g., for the harmonic oscillator, you have $\omega$ as a parameter, as well as the masses of the particle(s) involved, say $m$, and you also have time — even though it's not something that shows up explicitly in the Hamiltonian (remember explicit time dependency from Classical Mechanics: Poisson Brackets, Canonical Transformations, etc — in fact, you could get your answer straight from these kinds of arguments).
In this sense, just like you don't have a 'transformation pair' between $m$ and $\omega$, you also don't have one between time and Energy.
What do you say to convince yourself that $\omega \neq -i\, \partial_m$? Why can't you use this same argument to justify $E \neq -i\, \partial_t$? ;-)
I think Roger Penrose makes a nice illustration of how this whole framework works in his book The Road to Reality: A Complete Guide to the Laws of the Universe: check chapter 17.
The energy spectrum is bounded from below. A time operator would contradict the Stone–von Neumann theorem. This isn't really a problem. All it means is we have limits as to how accurate clocks can be in quantum mechanics.
The time which an event occurs, like the position it occurs at is an observable. Generally, QM stipulates all observables should be Hermetian operators. But QM actually does not construct an operator for time.
One can argue that since spacetime in QM is Newtonian and time there has a special status then this exception is well-motivated. However, relativity suggests that time should be understood similarly to space. This means in relativistic QM either time has to be promoted to an operator or position has to be demoted from being understood as an observable. The first significant relativistic QM theory was Diracs equation which modelled a single spinning electron. Here, he took the second option: time is understood on the same basis as position - not as operators but as coordinates.
This was fine for a one-particle theory but problems again appear when we attempt to generalise to Diracs theory to two particles. Dyson wrote:
This kind of 2-particle Dirac equation is no longer relativistically invariant, if we give each particle a separate position in space but all the same time. To avoid this Dirac constructed the many-time theory in which each electron has its own private time coordinate, and satisfies its private Dirac equation. This theory is all right in principle. But it becomes hopelessly complicated when pairs are created and you have equations with new time-coordinates suddenly appearing and disappearing
The resolution of this problemtic is in QFT where following Feynman we choose spacelike surfaces between which we evaluate the quantum amplitude on histories between them - aka the path integral; and then following Schwinger we can get rid of the problematic histories by reformulating it as an action principle from which - according to Dyson - the main features of QFT fall out simply - for example, the commutation relations for the fields.