How do mirrors work?

The reflection could be viewed as a two step process. The incident wave causes the electrons in the silver to vibrate like in an antenna. Though by vibrating they also emit the same light. So it's the electrons at the surface of the silver that reflect the incoming wave. As you mentioned the wave is part electric and part magnetic, but these cannot be taken apart since they are each others cause and effect: without one the other wouldn't be there either, and therefore it must reflect both parts.

That silver (and all metals) don't distort is due to the fact that they are also very good conductors. This prevents the electromagnetic waves from entering the object. The boundary conditions which must hold (from being an conductor) result in the perfect reflection and that the resulting angle is equal to the incident angle.

Similar boundary conditions are there for non-conducting materials like plastic and glass. These similar conditions result in reflection of glass and the shine/reflection on other smooth surfaces (though there can be other causes too). Also Snell's law would follow from these boundary conditions.

In contrast to conducting materials it is possible for electromagnetic waves to enter non-conducting objects. As a consequence part of the incoming wave is transmitted into the material. The propagation or dampening of the wave through the material is largely dependent on the properties of the material. Some materials like glass hardly dampen the wave and you can see through them, while others like most plastics dampen them and thus are opaque.