How to uninstall a .NET language pack for good?

Okay, turns out it really was Windows Update being annoying. For some obscure reason, it thinks language packs are important updates.

What I did was disable automatic updates, checked for updates, and choose to permanently hide this update. I don't know how long that will work though!

Microsoft, if you are reading this: please don't reinstall stuff that was manually uninstalled for a good reason! That's unexpected behavior and makes for a pretty bad user experience.