How do I stop Windows from muting background applications?

It's not a Windows issue. Applications get notified on whether they hold the focus in the desktop. Some of them will "for your own convenience" stop emiting sound when they lose the focus.

But that's not Windows muting them, it's just that the apps stop playing.

Easy test: If you open a video in your browser, minimize it and do something else, the video will still be played and it will still sound.

Usually games do a lot of things they wouldn't be normally doing, such as reshaping your mouse pointer or even hiding it and drawing another thing that does not really point to the OS-advertised pointer (such as Skyrim) that are even more a pain to run in a windowed environment.

And those games do all those kind of things without that many options screens, so it's unlikely that you'll be able to check whether you want to mute them or not when they are not holding the focus.