Why $SU(3)$ and not $U(3)$?

Suppose that $\text{U}(3)$ was the gauge group. We can decompose this as $$\text{U}(3)=\text{U}(1)\times\text{SU}(3),$$

which implies that in addition to the $\text{SU}(3)$ that has eight generators corresponding to eight gluons, there would be an additional generator for $\text{U}(1)$. The latter in principle corresponds to an additional gauge boson, but a theory of the strong interactions containing such a particle is inconsistent with experiment.