how to underclock an nvidia GPU on Windows?

You might try NVIDIA nTune :

NVIDIA® nTune is the ultimate utility for accessing, monitoring, and adjusting your system components, including temperature and voltages with clear, user-friendly control panels. Overclock your system for highest performance or underclock it for near silent operation. All changes are performed within the Microsoft® Windows® interface – enabling full functionality without the need to make changes in the BIOS and reboot your system.