Global symmetries in quantum gravity
Perhaps this is just rephrasing your last explanation, so I am not sure if you consider this as a "better argument", but I'll give you a good reference for further reading.
Quantum gravity may break global symmetries because the global charge can be eaten by virtual black holes or wormholes, see this paper.