Does installing a bunch of unnecessary fonts slow computers down?
The reason that it is recommended is that Windows (at least, not sure about Mac and Linux) has to load font information on boot. Not only can this slow the boot process (though I don't think you would actually notice this on a modern computer) but, more importantly, each font requires an amount of in-memory storage. This is then no longer available to other OS processes and so can slow down the OS due to paging.
So on a modern computer with plenty of memory and a fast drive, you won't notice any difference under normal circumstances.