Conceptually, what does unitization do?

Unitization and metric completion are both left adjoint functors, as are may other "-tion" operations in mathematics, such as localization or abelianization. Specifically, there is a forgetful functor from unital algebras to nonunital algebras (including norms is not particularly important here), and unitization is its left adjoint. Conceptually this means that it is in some sense the "freest" or "laziest" way of adding a unit to a nonunital algebra. (It's important to keep in mind that "nonunital" means "not necessarily having a unit" here.)

An important difference between unitization and metric completion is that the inclusion of complete metric spaces into all metric spaces is fully faithful, but the inclusion of unital algebras into nonunital algebras is not. This is responsible for your observation that when you complete a complete thing nothing happens, but when you unitize a unital thing you get something different. One of the relevant keywords here is idempotent monad.

One way of thinking geometrically about what unitization accomplishes is to apply the commutative Gelfand-Naimark theorem to nonunital C*-algebras. This says that every nonunital C*-algebra is the algebra of functions vanishing at infinity on some locally compact Hausdorff space $X$. Taking the unitization then gives the algebra of functions on the one-point compactification $\hat{X}$ of $X$. Taking the unitization again gives the algebra of functions on the one-point compactification of $\hat{X}$, which is $\hat{X}$ with a disjoint point added.

I don't remember where I read this, but Gert Pedersen once said something to the effect that "When I was young, the first thing we did with any C*-algebra was to adjoin a unit, but nowadays the first thing we do is remove the unit by tensoring with the compacts." The point is that $\mathcal{A}\otimes \mathcal{K}$ is the "stabilization" of $\mathcal{A}$: it is isomorphic to the $n\times n$ matrices over itself, for any $n$.