Why is name mangling not standardized

The Standard doesn't actually require name mangling. For that matter, the Standard doesn't require IEEE floating point numbers, or any number of other things.

Until there was a widespread ABI it could rely on, GCC actually went out of its way to use a different name mangling scheme than its competitors:

G++ does not do name mangling in the same way as other C++ compilers. This means that object files compiled with one compiler cannot be used with another.

This effect is intentional, to protect you from more subtle problems. Compilers differ as to many internal details of C++ implementation, including: how class instances are laid out, how multiple inheritance is implemented, and how virtual function calls are handled. If the name encoding were made the same, your programs would link against libraries provided from other compilers--but the programs would then crash when run. Incompatible libraries are then detected at link time, rather than at run time.

Name mangling is also more complex than many programmers realize. For instance how would the Standard specify acceptable calling conventions for all platforms that C++ could run on? Should a RISC system be required to support x86 stdcall even though RISC systems usually pass their arguments in registers instead of on the stack?

The standard does not address implementation details. There are many, many things which depend on the implementation, and which prevent programs from working together: how the classes are laid out, the structure of the vtable, etc. In general, compilers will change the name mangling if they change any of these. This is intentional, as it prevents code which would not work from linking.

It is possible for a given platform to define a C++ ABI; all compilers which adhere to it would use compatible implementations, and have a common name mangling. This is an issue for the platform vendors, however; for whatever reasons, very few vendors have defined a C++ ABI.

And the reason extern "C" works is because almost all platforms define a C ABI.