How can I hide certain playback devices in the Windows 10 taskbar?

Disabled audio devices should not show in that list.

If you are never going to use a given audio device, then in the Control Panel open the Sound (or Manage Audio Devices) item.

For any entries you do not use, right-click the item in the list and select Disable.