Why can I see a diamond?

Light does not have to make outer shell electrons leap the full band gap to interact with them. Electrons can be excited to virtual states, whence photons of the same energy and momentum are emitted. So although it is true that the absorption loss for pure diamond is very small as you've rightly inferred, a phase delay arises from this interaction, as I discuss further in this answer here.

In diamond, this phase delay is big: diamond has a refractive index of about 2.4 for visible light. So you see all the effects of the strong difference between the diamond's refractive index and that of the air around it: you see a diamond plate shift transmitted light sideways relative to the background, you see a strong specular reflexion from surfaces (the power reflexion ratio is about 17% for diamond) and, for white light, you see strong dispersion into colors for glancing reflexions and transmissions.

My take:

Photons have a wave function that carries the electric and magnetic field information which is a solution of the quantized Maxwell's equation. Thus there exist phases between photons and the superposition is again a wavefunction which macroscopically builds the electric and magnetic fields. Lubos Motl has a blog entry on how this happens at a QFT level.

When a photon hits a boundary condition , three things can happen: a) it can scatter elastically, which means it retains its frequency but changes angle, b)it can scatter inelastically, which means it changes frequency, or c) it can be absorbed raising the energy level of an electron ( in a lattice, in a molecule, in an atom) and a different photon is emitted and phases are lost.

For a reflective surface where images are retained a) happens: all phases in the emergent ensemble of photons are intact.

For an opaque surface c) happens

For a transparent lattice it is still a). The photon interacts elastically with the lattice, phases through the ensemble are kept coherent and thus we see through glass. It is a "photon + lattice" scattering at an individual level, but for a medium to be transparent the emergent ensemble of photons must retain coherence. Phases change coherently in the quantum mechanical solution, otherwise there would be no transparency

In a diamond there are reflective surfaces that back scatter part of the light , images are distorted but still phase information is coherent.

b) is the case where colors change if the scattering is with the whole lattice and phase coherence can be kept.

The Kramers-Kronig relations connect the real part of the permittivity $\epsilon_r(\omega)$ to the imaginary part $\epsilon_i(\omega)$ [or real and imaginary refractive index, $n_r(\omega)$ and $n_i(\omega)$], i.e., the phase velocity of light in a crystal at a given frequency to the absorption properties over the whole frequency range. Thus the high refractive index of diamond in the visible region is related to the absorption bands in the UV region related to the band gap.