How does non-Abelian gauge symmetry imply the quantization of the corresponding charges?

First of all, note that the real Abelian Lie group $U(1)$ comes in two (multiplicatively written) versions:

1. Compact $U(1)~\cong~e^{i\mathbb{R}}~\cong~S^1$, and

2. Non-compact $U(1)$ $~\cong~e^{\mathbb{R}}\cong~\mathbb{R}_+\backslash\{0\}$.

Also note that in the physics literature, we often identifies charge operators with Lie algebra generators for a Cartan subalgebra (CSA) of the gauge Lie algebra.

Moreover, note that the choice of CSA generators is not unique, see also this answer. The ambiguity in the convention choice of charge operators is similar to the ambiguity in the convention choice of spin operators, see also this question. We shall from now on assume that we consistently stick to only one such possible convention.

Given a Lie algebra representation, the eigenvalues of the charge operator are called charges.

Now let us briefly sketch some lore and facts related to OP's question (v2).

1. We observe in Nature that Abelian and non-Abelian charges are quantized, as accurately described by electric charge, electroweak hypercharge, electroweak isospin and color charges in the $U(1)\times SU(2)\times SU(3)~$ standard model.

2. If there exist dual magnetic monopoles, then quantum theory provides a natural explanation for charge quantization. Namely, by playing with Wilson lines, the singlevaluedness of the wavefunction requires that charges are quantized (i.e., to take only discrete values), and that the gauge group is compact, as first explained by Dirac.

3. It is a standard result in representation theory, that for a finite-dimensional representation of a compact Lie group, that the charges (i.e., the eigenvalues of the CSA generators) take values in a discrete weight lattice.

4. If a gauge group contains both a compact and a non-compact direction, i.e. if its bilinear form$^1$ has indefinite signature, it is impossible to define a non-trivial positive-norm Hilbert subspace of physical, propagating, $A^a_{\mu}$ gauge field states.

--

$^1$ By a bilinear form is here meant a non-degenerate invariant/associative bilinear form on the Lie algebra. For a semisimple Lie algebra, we can use the Killing form.

The situation is similar with SU(2) spin quantization. Generators of the SU(2) are quantized, while U(1) this is not the case. Spin is quantized in 3D space, but in a 2D space it is continuous real number, with fractional quantum statistics intermediate between boson and fermion.