Why zsh and ksh93 chose to be non-compliant in pattern matching?

The passage you quote does not mean what you say it means.

Patterns Matching a Single Character

(…) An ordinary character is a pattern that shall match itself. (…) If any character (ordinary, shell special, or pattern special) is quoted, that pattern shall match the character itself.

All of this applies only to characters that stand for themselves in a pattern. This does not apply to characters that appear in a context other than that where a pattern character is expected. In particular, it does not apply inside a bracket expression. The syntax of bracket expressions is described under the entry for [:

If an open bracket introduces a bracket expression as in XBD RE Bracket Expression, (…)

(I omitted the bit about ! vs ^ for complementation.) The description of RE bracket expressions doesn't say anything about quoting (which is unsurprising since it's about bracket expressions in general, not about bracket expressions in a pattern in a shell script).

Going by a strict interpretation of POSIX.1-2008, it isn't clear what the pattern ["!"a] should match. One interpretation is that it should match any of the characters ", ! or a: the character " has no special meaning inside a bracket expression. I can't find anything in the specification that would invalidate this interpretation. Another interpretation is that " retains its quoting behavior, but that means that the content of the bracket expression is !a, and since there is no particular treatment of quoted characters inside bracket expressions, the set is all-but-a. I can't find any support for your interpretation (and the behavior of dash, bash and other shells) in the POSIX specification. It makes sense, sure, but it isn't what the text says.

It would make sense for a future version of POSIX to mandate your interpretation, by adding some wording to this effect. For example, the description of [ could be changed to

If an open bracket introduces a bracket expression as in XBD RE Bracket Expression, except that the \ character ('!') shall replace the \ character ('^') in its role in a non-matching list in the regular expression notation, it shall introduce a pattern bracket expression, and that any character that is quoted shall stand for itself as an element of the enclosing bracket expression, collating element or class expression. A bracket expression starting with an unquoted \ character produces unspecified results. Otherwise, '[' shall match the character itself.

Given that POSIX is mostly descriptive rather than normative, I'd expect such a change that breaks ksh (usually the reference shell) to only be included in a major update of the standard, and any defect on the existing version to instead allow at least the existing different interpretations.

That's a bug in zsh, which was reported in this discussion [BUG] quoting within bracket patterns has no effect:

case b in
  ( ['a-c'] ) echo 'false match' ;;
  ( [a-c] )   echo 'correct match' ;;
esac

will print false match instead of correct match.

The fix is scheduled to be released with zsh version 5.3.