How can vacuum have a permittivity?
That is a misnomer due to confusion caused by the adoption of SI units. epsilon naught has nothing to do with permittivity or the vacuum. EM was developed in cgs units. 1/4piepsilonnaught is just a factor for the conversion of units, and has nothing to do with physics. You might recognize from the 9 in its numerical value that it is just c^2 in mixed up units. The c^2 is necessary because of a mismatch in the original units between magnetic and electric units.
In SI units, permittivity scales the strength of the electric field relative to the charge density, via:
$$ {\bf \nabla\cdot E} = \rho/\epsilon_0 $$
When we introduce linear media, we find:
$$ {\bf \nabla\cdot E} = \rho_{free}/\epsilon $$
where $\rho_{free}$ is the free charge, excluding the charge from polarization of the media, ${\bf \nabla \cdot P}$.
So it's not that the vacuum is being polarized in the media free case, it's that linear media, through polarization, acts to weaken the coupling between free charge and electric field.
The constant $\epsilon_0$ almost never appears in an expression where it's added to another term; it's multiplied instead. Contrast with electric charges, which combine by addition, and for which the "natural" value is zero. For a multiplicative constant the "natural" value is one, though you can complicate the situation by introducing an unfortunate choice of units.
In quantum electrodynamics it becomes important to know the relation $$ \frac{e^2}{4\pi\epsilon_0} = \alpha \hbar c $$ where $e$ is the fundamental charge, $\hbar$ is the reduced Planck constant, and $c$ is the speed of light. You should choose your favorite system of units and do the arithmetic to confirm that the "fine structure constant" $\alpha$ is dimensionless and has a value $\alpha \approx 1/137$. The "small" value for the dimensionless $\alpha$ means that $\alpha^2 \ll \alpha$, and is the main reason we can use perturbation theory to describe the interaction between electric charges as "approximately" one-photon exchange, with "corrections" due to virtual particles and the like. In the mathematical formalism, each level of "correction" is less important by a factor of $\alpha$.
The equivalent dimensionless coupling for the strong interaction is $\alpha_s \approx 0.1$, and there are more types of gluons than there are types of photon, so we say that the strong interaction is "non-perturbative" and must use much less satisfying techniques to make predictions about it. The main result of this difference is color confinement, which is why our world is made of protons and neutrons instead of quarks.
A world in the limit $\epsilon_0 \to 0$ would be a world in the limit $\alpha\to\infty$, and would be as different from the world where we live as the strong force is from electromagnetism. It's better to think of $\epsilon_0$ as "one unit" of vacuum permittivity.