Why do you need PulseAudio?

It's all about multiplexing.

I don't need to multiplex sounds from or to several sound cards

Ah, but you do! If you want to be able to play audio from two sources at once, ever, for any reason, you need multiplexing.

OSX and Windows handle Multiplexing in the Kernel (but still in software), which is why this never/rarely comes up on those platforms.

However, on Linux, with software like ALSA, multiplexing is left up to the specific sound card / implementation / driver.

Unfortunately, not all cards and all drivers actually support this out of the box.

That's where PulseAudio comes in, doing the multiplexing in software, regardless of your sound card / driver situation.

Without this functionality, if you were say, using ALSA directly sans-PulseAudio, with a sound card that had poor PCM multiplexing support on Linux, you would only ever be able to hear sound from one application at a time.

E.g. if you had a video playing in your web browser, and received a notification in Pidgin, you would not hear the notification sound because your web browser would already have control of your sound card.

By routing all sound through PulseAudio first, this problem is avoided.

Source: A long IRC conversation I once had with the Ubuntu maintainer for ALSA, where I asked them the exact same question you're asking now.

Since introduction of Dmix in alsa, pulseaudio turned to be useless. Bare alsa with Dmix somehow enabled deep inside (I haven't have to set anything) works much better for me. For example, there is no delays while a sound level is being changed.

Just uninstall pulseaudio if you are using recent alsa and feel fine.

You don't. It's a piece of middleware that for most users is completely unnecessary. Most applications that need audio can use ALSA directly just fine. ALSA can handle things like basic multiplexing perfectly well (although it might possibly need a plugin). On my system, I don't have pulseaudio installed and I can play a video game and have music playing in the background from Rhythmbox, no problem. It works right out of the box with ALSA, no intricate setup required.

I would recommend to anyone that is experiencing any audio-related problems at all: first thing to try is tear out pulseaudio. I had some audio issues myself recently, took it out and they went away immediately.

Tbh, it's hard to think of a case where someone would actually need pulseaudio. For a 'typical' home system user that just wants the sound to work with their desktop applications, ALSA by itself is perfectly suitable. For a more advanced user that wants to do more complex audio tasks, or someone who needs professional quality audio, JACK is clearly what you want. Pulse, imo, seems to be rather bloaty and superfluous.