Why is taking the address of a destructor forbidden?
The other answers talked about the calling convention potentially differing, and this is one reason.
But I think also: It's not clear how useful taking the address of a destructor really would be! What would you use it for? Calling it is potentially very unsafe -- would you use it for equality comparisons? To determine the exact type of an object, perhaps? There are other language facilities for that.
Ultimately if the destructor is emitted as a function it does have an address -- but the language considers using that address for anything, even equality comparisons so terribly not useful it's just forbidden at the language level.
Technically the compiler could have allowed you to do it -- perhaps forcing it to a void * pointer or something -- but then it would have to be specified which address exactly you would be given. It's also not always the case that the compiler even emits a destructor for trivial classes as a function -- so taking the address would require it to do so.
It would only be useful for equality comparisons, potentially -- and defining what the equality guarantees are is probably not worth it since other language facilities can potentially achieve the same thing.
Tl;Dr: It's just considered unsafe and not useful, so the language forbids it.
Constructors and destructors are somewhat special. The compiler often uses different conventions when calling them (e.g. to pass extra hidden arguments). If you took the address and saved it somewhere, the compiler would lose the information that the function is a constructor or destructor, and would not know to use the special conventions.