Duplicate iptable rules
Solution 1:
List with line numbers and delete by number.
iptables --line-numbers --list
Then delete one rule using it's line number. Then repeat (line numbers change for following rules when one is deleted so re-list before deleting another).
iptables -D INPUT 6
Solution 2:
iptables-save | uniq | iptables-restore
That is all you need really.
Solution 3:
If you only want to delete double lines that are directly one after another you can export, unify and reimport it with
mkdir ~/tmp
iptables-save > ~/tmp/iptables.conf
uniq /tmp/iptables.conf > ~/tmp/iptables_new.conf
iptables-restore < ~/tmp/iptables_new.conf
If you want to delete other lines use an editor on ~/tmp/iptables.conf before you reimport it the same way.
Check your new rules with
iptables-save
Solution 4:
A comment to fail2ban: fail2ban seems to add its iptables rules itself. So you should not store these rules with e.g. iptables-save. Then after a reboot the rules will be doubled (your saved rule + the rule added by fail2ban).
Solution 5:
Delete all duplicated lines except COMMIT
/sbin/iptables-save | awk '!COMMIT||!x[$0]++' | /sbin/iptables-restore