Why is RFC 7505 (Null MX) necessary?
Solution 1:
Because that is not what you are supposed to be using .invalid for. Like .example it is meant for local testing and documentation.
Additionally, using .invalid still causes additional things to happen - additional DNS look ups and queuing on the mail server for retries for one off the top of my head.
Using the "." format is supposed to cause an immediate hard fail. Causing the MTA to immediately stop attempting delivery. At least that's the way the intro to the RFC reads.
Solution 2:
The question as a whole touches on a few different aspects which all need to be taken into consideration to answer why RFC7505 adds something useful.
First of all, the pre-RFC7505 definition of how mail delivery should be made does not have a way to cleanly indicate that no mail delivery attempts should be made for a name which has address records.
From RFC7505 section 1:
SMTP clients have a prescribed sequence for identifying a server that accepts email for a domain. Section 5 of [RFC5321] covers this in detail; in essence, the SMTP client first looks up a DNS MX RR, and, if that is not found, it falls back to looking up a DNS A or AAAA RR. Hence, this overloads a DNS record (that has a different primary mission) with an email service semantic.
If a domain has no MX records, senders will attempt to deliver mail to the hosts at the addresses in the domain's A or AAAA records. If there are no SMTP listeners at the A/AAAA addresses, message delivery will be attempted repeatedly for a long period, typically a week, before the sending Mail Transfer Agent (MTA) gives up. This will delay notification to the sender in the case of misdirected mail and will consume resources at the sender.
This document defines a null MX that will cause all mail delivery attempts to a domain to fail immediately, without requiring domains to create SMTP listeners dedicated to preventing delivery attempts.
Then there's the matter of how RFC7505 implements this (IN MX 0 .).
From RFC7505 section 3:
MX Resource Records Specifying Null MX
To indicate that a domain does not accept email, it advertises a single MX RR (see Section 3.3.9 of [RFC1035]) with an RDATA section consisting of preference number 0 and a zero-length label, written in master files as ".", as the exchange domain, to denote that there exists no mail exchanger for a domain. Since "." is not a valid host name, a null MX record cannot be confused with an ordinary MX record. The use of "." as a pseudo-hostname meaning no service available is modeled on the SRV RR [RFC2782] where it has a similar meaning.
A domain that advertises a null MX MUST NOT advertise any other MX RR.
(emphasis added)
As is noted here, the implementation details for the "null MX" are based on an already established pattern from the SRV RR type. It makes sense to mimic this as the SRV RR type more or less is a generalized version of the MX RR type.
So, the decision was essentially taken already when defining the SRV RR type.
So, why not make use of .invalid?
From RFC2606 section2:
".invalid" is intended for use in online construction of domain names that are sure to be invalid and which it is obvious at a glance are invalid.
As can be seen here, this reserved TLD is for human consumption. There's no precedent of defining special handling of this in software.
Surely it could have been implemented in some different way but they chose to go with the minimal approach of using ., which is not a valid hostname and thus does not interfere with normal use anyway.