Verifying MTA for correct behavior with swaks
Your example is not sufficient to show that plaintext auth isn't allowed over non-tls connections. --protocol SMTP
is explicitly telling swaks NOT to use ESMTP, and ESMTP is required for authentication. In other words, you're testing that auth isn't offered over SMTP, not that plaintext auth isn't offered over plaintext connections. (who's on first!).
The following is closer to what you're looking for:
# These should fail, because you don't want to offer plaintext auth protocols
# over non-tls connections
swaks ... --auth PLAIN --auth-user .. --auth-password ..
swaks ... --auth LOGIN --auth-user .. --auth-password ..
# Should succeed because hashed PW protocols are ok over plaintext (assuming you
# support them at all of course):
swaks ... --auth CRAM-MD5 --auth-user .. --auth-password ..
swaks ... --auth DIGEST-MD5 --auth-user .. --auth-password ..
swaks ... --auth NTLM --auth-user .. --auth-password ..
# The converse of the above, make sure your plaintext password work over tls
# sessions (assuming you want them to, of course)
swaks ... --auth PLAIN --auth-user .. --auth-password .. --tls
swaks ... --auth LOGIN --auth-user .. --auth-password .. --tls
swaks ... --auth CRAM-MD5 --auth-user .. --auth-password .. --tls
swaks ... --auth DIGEST-MD5 --auth-user .. --auth-password .. --tls
swaks ... --auth NTLM --auth-user .. --auth-password .. --tls
Hope that helps, good luck!