How to remove unwanted noise peaks in sound files in Windows?

Have you tried Audacity's Noise Removal feature(s)?

This may be more of a 'click' than a 'noise' (looks like it), so perhaps you'll have better luck with the Audacity "Click Removal" features?

Example of what's on the page:

Sometimes an even better result can be obtained by zooming in (CTRL + 1) to near sample level and either silencing the click (Edit > Silence), or using the Draw Tool to smooth out the contours of the samples and so attenuate the click.

Truncating peaks like that can be done using the Limiter (Effect > Limiter).

You probably want to use a "Hard Limit", as only to affect the peaks. Also, you can of course use it multiple times in combination with Effect > Normalize to shrink those peaks while scaling up the remainder of the recording to cover the full volume range.