MP4 video will not play in Internet Explorer 11
So this is somewhat embarrassing, but I did eventually figure it out.
The problem wasn't so much with Internet Explorer as it was the VM I was using to run Windows to test in Internet Explorer.
Essentially, I was running Windows 7 inside of a VirtualBox VM. When I (in desperation) sought out an actual physical machine running Windows... lo and behold it played!
I can only assume that the video drivers provided by VirtualBox didn't like decoding MP4 or something. The videos played fine in other browsers, but I'm guessing they decode/render video differently to how IE does.
I had the same issue recently with VirtualBox running Windows7 on MacOS. Solved by enabling 3D acceleration in the "Display" settings for the problematic VM.