What is the significance of taking the commutator?

Uncertainty and Measurements

Suppose the operators $A$ and $B$ are Hermitian and represent observables. Then if we make respective measurements in an experiment, then the standard deviation in $A$ denoted $\sigma_A$ as well as in $B$, $\sigma_B$, must satisfy the inequality,

$$\sigma_A \sigma_B \geq \frac12 |\langle [A,B]\rangle|.$$

Thus, two operators not commuting has an implication on the limitations of measurement, that is, if we make some measurement with $\sigma_A$, there is a limit to how well we can measure $B$.

---

Constants of Motion

A further application of the commutator is to determine conserved quantities. If $Q$ is to be conserved, it must commute with the generator of time translations which is the Hamiltonian, and thus we require $[Q,H] = 0$. This follows from Ehrenfest's theorem,

$$\frac{d}{dt}\langle Q \rangle = \langle \partial_t Q\rangle - \frac{i}{\hbar}\langle [Q,H]\rangle.$$

---

Canonical Quantisation

As noted in a comment, the analogue of the commutator in classical mechanics is the Poisson bracket, which has certain implications, and an interpretation in terms of a symplectic manifold known as the phase space of a system.

In quantum mechanics, as well as relativistic quantum mechanics, one can pass from a classical theory - at least in theory - to a quantum theory through canonical quantisation by changing the Poisson bracket to a commutator, as a rule of thumb:

$$\{A,B\} \to -\frac{i}{\hbar}[A,B].$$

There are many subtleties - too many to list here - as to how this can go wrong and need further refinements to quantise a theory. However, a common one is ordering ambiguities. Recall that in a classical theory we do not deal with operators and do not have to worry about ordering, which may differ by a commutator. In quantum mechanics, we do and require an ordering prescription.