Does fail2ban do Windows?
Solution 1:
I know of no tool that will do this "out of the box". I wrote a script to do something like this with failed OpenSSH logons on Windows, but I can't share it with you because it "belongs" to the Customer for whom I wrote it.
Having said that, it was a simple VBScript program that had an event log sink to watch for new failed logons and, if enough happened in a time window, add an IP route (using the "route" command) to route traffic to the offending IP address to a "MS Loopback Adapter" on the system.
For other types of logs, it would be a fairly trivial matter to write. Since I didn't have IPtables on Windows, the loopback adapter seemed like the next best thing. (You can't do a "route x.x.x.x mask 255.255.255.255 127.0.0.1" on Windows-- you need an adapter to route the traffic to, because the 127.0.0.1 loopback isn't a "real" interface on Windows.)
(If you want something like this written, contact me out-of-band and we can discuss the specifics of such an arrangement.)
Edit:
I decided to write something to do this and I've released it under a Free license.
Solution 2:
Check out this project - ts_block
I'm using it and thus far its terrific (Windows Server 2008 R2 RDS, system is behind a firewall but I didn't feel like using an ssl vpn gateway to the server)