Does the magnitude of a physical quantity have units or is it just a plain number?

The magnitude has units. In your example, it's physically how fast you're going, which is measured with units. It doesn't make sense to say you're going "36", and so it doesn't make sense to say the magnitude of your velocity vector is 36.

Saying that the magnitude is 36 is a bad idea, because if you measured in cm/s instead, the magnitude would be 3600, and the magnitude would change depending on what units you had. Instead, we attach units to the magnitude so it can be expressed as 36 m/s or 3600 cm/s, but these are the same quantity, so the magnitude doesn't change with different units. It's a property of the vector, not an accident of the units chosen.

It makes sense to assign the units to the magnitude and not the direction vector, but it would work either way.

Consider the position vector denoted by $$ \boldsymbol{r} = \pmatrix{ 3\, {\rm m}\\ 2\, {\rm m}\\ 6\, {\rm m}} = \pmatrix{3\\2\\6} {\rm m} $$

The magnitude of the vector is $ \| \boldsymbol{r} \| = 7 {\rm m} $, but to decompose it into magnitude and direction we have a choice:

$$ \boldsymbol{r} = (7 {\rm m}) \,\pmatrix{ \frac{3}{7} \\ \frac{2}{7} \\ \frac{6}{7} } = (7) \,\pmatrix{ \frac{3}{7}\,{\rm m} \\ \frac{2}{7}\,{\rm m} \\ \frac{6}{7}\,{\rm m} } $$

• The first being the distance $7 \,{\rm m}$ in the direction $\left(\frac{3}{7} , \frac{2}{7} , \frac{6}{7} \right)$. This is the span interpretation when one spans "x" distance along a particular line.
• The second being 7 times the distances $\left(\frac{3}{7}\,{\rm m} , \frac{2}{7}\,{\rm m} , \frac{6}{7}\,{\rm m} \right)$. This is the ruler interpretation, where one moves "x" number of ticks in more direction and each tick has units.

In fact, there are cases where both the magnitude and unit vector may contain units. For example, consider a planar force acting along a line. Now combine the force components with the equipollent torque at the origin.

$$ \boldsymbol{f} = \left[ \matrix{ \vec{F} \\ \tau } \right] = \left[ \matrix{6.9282\,{\rm N} \\ 4.0 \,{\rm N} \\ 20.0 \,{\rm N\,m}} \right] = (8.0\,{\rm N}) \left[ \matrix{ 0.866 \\ 0.5 \\ 2.5\,{\rm m} } \right] = F \,\hat{\ell} $$

The magnitude of the force is $F = 8\,{\rm N}$, acting along a line with equation $a y - b x + c =0$ where $\hat{\ell} = (a,b,c) = (0.866, 0.5, 2.5)$ and the direction vector $(a,b)$ has unit magnitude $\sqrt{0.866^2+0.5^2}=1$ and thus $c=2.5$ is a distance quantitiy (for $ay-bx+c=0$ to be dimensionally accurate).

You would quote the speed as $36 \, \rm m\,s^{-1}$ because it is $36$ times $1 \, \rm m\,s^{-1}$.
If you wrote just $36$ what would that mean?
$36$ times what ????

Now what about $36 \, \rm m\,s^{-1}\, \hat u$?
All you have now is extra information about the direction of the velocity and there is no extra information about the magnitude (speed) which is still $36 \, \rm m\,s^{-1}$.