Is it possible to tell windows 10 to not take all the available bandwith when downloading updates?

There is a way to throttle Windows Update in Windows 10 v1703 or higher (without third party software).

Go to Settings (Win+I) > Update & Security > Advanced options > Delivery Optimization > Advanced Options. You should see this.

Download Settings

Limit how much bandwidth is used for downloading updates in the background.

Turn it on, and set it to something low (I have it set to 25%).

If you want to see how these settings affect your download speeds, go to Settings (Win+I) > Update & Security > Advanced options > Delivery Optimization > Activity monitor.

This fixed my problem with Windows Update, and will hopefully fix yours too.