DNSSEC signing algorithms
(Yes, the question is old, but it deserves an answer.)
Some rules for modern crypto:
- MD5 is dead
- 1024 bit RSA is dead
- SHA1 is dying
Using an HMAC for DNSSEC makes no sense, an HMAC requires both parties to have access to the same secret; in the context of DNSSEC, this means clients could spoof the server, making it useless.
There's a lot of algorithms missing from your list, I don't know why Virtualmin gives you those options. ECDSA options give you smaller responses and somewhat lower CPU usage in signing operations.
Given the relatively short lifetime of DNSSEC keys & responses, I would favor shorter options after taking all of the above in to account. This means the use of RSASHA256 with a 2048-bit RSA key or ECDSAP256SHA256 (256-bit ECDSA with SHA256).