What really are units? And why is it valid to ignore them (once you have dimensional homogeneity), as is done in class?
We can think of units in a formal sense: they can be multiplied and divided, but different units cannot be added. In one way, this is like working in a space of real numbers where each axis represents a power of a dimension, those axes are closed under addition, and that multiplication sends us to a different axis (i.e. you can do meters + meters, but not meters + meters-squared). So in a sense, you could form a basis from the fundamental units and their powers -- actually, this is pretty much what is happening in the Buckingham Pi theorem! Of course, we have a unitless basis element as well.
Nevertheless, it's not really worth worrying about. As I said, we needn't worry about $\mathbb{C}^5$ if we're trying to compute $22 \times 58$. So as far as profound reasons go, that's about as profound as I can make it -- the terms dimension and unit are used for precisely that reason. Operating on them allows us to work in a dimension with a unit basis where certain very-familiar rules hold.
Another analogy: if you walk 5 meters in one direction, you walk 5 meters. If you walk 5 meters in a different direction, you walk 5 meters. It may be eventually important to know whether you're walking east or south, but for the scope of simply wanting to know how far you've walked, we need not even care that east or south exist!
Consider the set of $1\times1$ matrices with real entries. We know how to add these. $[a]+[b]=[a+b]$. We know how to multiply them. [a][b]=[ab]. We can do division, subtraction, exponentiation, etc. It turns out they behave just like the real numbers! It seems that the only difference is that we put some brackets around them. (Turns out they're isomorphic fields.)
Now what if I asked you to compute $5+[8]$? The question doesn't make sense. You're adding a real number to a matrix. We haven't defined a way to do that.
Units are a bit like the brackets on a $1\times1$ matrix; they're bits of notation that don't change the underlying logic. And just as it makes no sense to add a matrix to a scalar, it makes no sense to try to add $5m+8s$.
I expect it's involved (as well as counterproductive to dispelling your sense of unease) to give a general answer, and impossible to give a non-controversial account. But assuming I've understood your question, here's one way to think of things:
There's a mathematical notion of the real number system (a complete ordered field), whose properties are independent of measurements or physics (in the sense that mathematicians specify axioms and deduce properties; of course, the axioms themselves are motivated by physics/experience).
There are physical notions (length/spatial displacement along a particular direction, area, mass, duration, ...) corresponding to measurable quantities that, in an idealized sense, behave like real numbers, e.g., can be added/concatenated/agglomerated or compared; or under appropriate conditions can be multiplied, as when multiplying two orthogonal lengths to obtain an area.
A choice of units for a particular quantity is a "mapping" or "single-valued association" from a physical notion (e.g., a length) to a mathematical one (a real number). (A single ruler may be both $12$ inches and $30.48$ centimeters in length.)
Now, in order to get meaningful physical interpretations from formulas, the units associated to terms have to be chosen compatibly. After all, you cannot add $x$ inches to $y$ centimeters and get $(x + y)$ of some length unit independent of $x$ and $y$. (We could agree that $5$ inches plus $3$ centimeters is $8$ "bugs", but this "definition" of "bug" would depend on the particular addends, not just on the particular units; $5$ inches plus $4$ centimeters would not be $9$ bugs.)
Similar considerations of unit compatibility hold whenever you compute physical quantities using mathematical theorems. Mathematical operations correspond to physical operations when, and only when, units are chosen compatibly.