What is the difference between electrons and holes in silicon?

One qualitative way to understand this concept is by thinking about the orbital origins of the conduction band and valence band.

For example, let's assume the conduction band is primarily coming from hybridized $s$-orbitals, while the valence band comes from hybridized $p$-orbitals. As you might guess, particles in the $s$-orbital should move differently from the $p$-orbital, as they both have different spatial properties ($s$ is more delocalized compared to $p$, and $p$ is anisotropic unlike $s$).

Now it should be clear that a hole living in the valence band should behave differently compared to an electron in the conduction band-it is because they are not simple inverses of each other, but instead occupy different bands! In fact, for most semiconductors the electron and hole transport properties are quite different, with a common exception being Graphene.