How can I reset or lower the serial used in BIND DNS server's SOA record?

If you have shell access to all slave servers, there is a simpler solution:

To be able to decrement a serial number on slave servers, you can log into the slave server and run rndc retransfer my.zone.name. This will cause the slave to fetch the zone from the master and completely ignore the serial number.

"BIND's internal libraries" don't care what the serial number is. It's only agreement between the master server and slave servers that matters. In other words, BIND will happily let you decrease the serial number in a zone file without complaint.. It's just that the slaves would no longer receive updates.

Zone file serial numbers are unsigned 32-bit integers and they wrap around the largest possible 32-bit unsigned integer. So there is a way to decrease the serial number by incrementing it repeatedly until it rolls over and becomes closer to zero. There is a maximum amount by which you can increment it at a time, so you have to do this iteratively in multiple steps:

• Increase the serial number by a large increment but no more than 2147483647
• Wait for all of the slave servers to catch up and be up to date with the current SOA.
• Repeat

You can always pick an increment such that you don't need to iterate more than twice.

Follow this HOWTO.