Physical dimensions for the Coulomb force in SI and CGS
Short Answer
You've hit upon the quirk that the SI and CGS systems not only measure electric charge with different units, but also assign them different dimensionality.
In SI, the Ampere is a base unit. Amperes are not made out of anything else - they are primitive, like meters, kilograms, and seconds. One Ampere is one Coulomb per second, so the unit of electric charge, the Coulomb, is equal to one Ampere-second.
When there's an equation that has Amperes or Coulombs on one side, and something without those units (like force) on the other side, the equation will always have a constant that has the correct units to balance things out. That job is done by $\mu_0$ and $\epsilon_0$.
In CGS, charge is measured in esu, but this is a derived unit. Charge is considered to be made up out of length, mass, and time the way, for example, angular momentum is. Charge has dimensions
$$[M]^{1/2}[L]^{3/2}[S]^{-1}$$
and one esu is equal to the square root of a $\text{dyne-cm}^2$.
Longer Answer
In SI, the we start with meters, kilograms, and seconds. Then we set $\mu_0$ to be $4\pi\times 10^{-7}\textrm{ohm-sec/m}$. Now take two wires very long compared to the distance between them, and run the same current through them. Make the distance between them $d$. A section of length $l$ of the wires will feel a force of attraction $F$, which depends on the square of the current. The Ampere is defined such that when the current is measured in Amperes, we find that
$$F = \frac{\mu_0 I^2 l}{2\pi d}$$
when $F$ is measured in Newtons, $d$ and $l$ in meters.
This definition requires that you can create the same current in each wire, that you can accurately measure the force per unit length, and that the force is perfectly proportional to the square of the current. In truth it won't be because the wires aren't infinitely long and perfectly straight and parallel, but some equivalent operational definition might be used in practice. (The fact that the definition is at all possible is a test of the physical hypothesis of the proportionality.) The important point is that the Ampere becomes a new basic unit. (Technically, what's been said so far would not define Amperes, but give us a relationship between Amperes and Ohms. We could hash that out by looking at the units of $V=IR$, for example).
The Coulomb used in Coulomb's law is then defined as one Ampere-second. This would seem to allow us to experimentally measure $\epsilon_0$ because it is now the only unknown in Coulomb's law. Originally, that was correct, but now we have defined the speed of light to be $c = 2.99792458 \times 10^8$ meters/second, and so we're constrained by $c = 1/\sqrt{\mu_0\epsilon_0}$. This forces $\epsilon_0$ to be
$$\epsilon_0 = \frac{1}{4\pi\times 8.9875517853681764 \times 10^9}\frac{\textrm{sec}}{\textrm{ohm-m}}$$,
meaning that we could also define the Coulomb directly from Coulomb's law - take two charged bodies, measure the force between them, and define the Coulomb to be the unit of charge such that
$$F = \frac{Q_1Q_2}{4\pi\epsilon_0 R^2}$$
with $R$ in meters and $F$ in Newtons. The experimental observation that these two definitions of the Coulomb (one as the Ampere-second and one directly from Coulomb's law) agree then becomes a test of the physical theory of electromagnetism.
In CGS units, charge is measured in esu, a derived unit defined by Coulomb's law as you've written it. One esu is the charge such that two charged bodies feel the Coulomb force
$$F = \frac{Q_1Q_2}{R^2}$$
with $F$ in dynes and $R$ in centimeters. If both charges are one esu, this gives
$$1\text{ }\textrm{dyne} = \frac{1\text{ }\textrm{esu}^2}{1\text{ }\textrm{cm}^2}$$
or
$$1\text{ }\textrm{esu} = 1\text{ }\textrm{cm}\sqrt{\textrm{dyne}}$$
finally, since the dyne is a derived unit, this could be written in terms of base units as
$$1\text{ }\textrm{esu} = \sqrt{\frac{\textrm{g-cm}^3}{\textrm{s}^2}}$$
As a consequence, there is one less base unit involved in electromagnetic formulas when using CGS. Also, there is no need for the constant $\epsilon_0$ used in SI. There is also no need for $\mu_0$. Instead, things like Maxwell's equation explicitly include the speed of light $c$. You can see a side-by-side comparison of basic equations of electromagnetism in SI and CGS units here.
Converting CGS centimeters and grams to SI meters and kilograms, then equating the two expressions for the force in Coulomb's law, we find the conversion that one Coulomb is the same amount of charge as $2.99792458 \times 10^9$ esu.
Reference
This answer is a summary of an appendix to Purcell's Electricity and Magnetism
Yes the dimension is different. In SI the current (A) a base unit independent from length (m), mass (kg) and time (s) because we choose to, but in CGS Gaussian unit this is not (1 unit of current = 1 g1/2 cm3/2 s-2), by setting $\epsilon_{0,SI} = \frac1{4\pi}$. This also leads to some perhaps unintuitive results, like the unit capacitance in CGS Gaussian is "centimeter", and resistivity is "second".
The unit system is actually arbitrary. We could, for example, make every physical observable a unitless number by setting $c = \hbar = k_B = \frac1{4\pi\epsilon_0} = G = 1$, and we get the Planck's natural unit, although those numbers become inconveniently large or small. This is also the reason of introducing ampere (and volt and ohm) as separate units, as that "unit of current"'s size is impractically small in everyday life (1 unit of CGS current is 3 × 10-8 A).