What's the interpretation of a unitary matrix?
Structurally, unitary matrices are rotations and reflections. Perhaps it's more clear to first picture unitary diagonalization before the singular value decomposition. Suppose we unitarily diagonalize $$A = UDU^{\dagger}$$ In unitary diagonalization, we first rotate (and possibly reflect) from our standard basis into our new orthonormal basis. This is the action of $U^{\dagger}$. Then we perform stretches by the magnitudes of the eigenvalues in the respective basis directions. This is the action of the diagonal matrix $D$. Finally we rotate back to our original basis, which is the action of $U$ which reverses $U^\dagger$.
The action of a singular value decomposition is virtually identical, except that the "diagonal" matrix $\Sigma$ does not necessarily map the same space to itself, so that the rotations happen in different vector spaces.
Unitary matrices are the complex analogues of orthogonal matrices, and both are very common in the theory of Lie groups and Lie algebras. Orthogonal matrices are the matrix representations of real linear maps that preserve distance. Unitary matrices are the complex versions, and they are the matrix representations of linear maps on complex vector spaces that preserve "complex distances".
If you have a complex vector space then instead of using the scaler product like you would in a real vector space, you use the Hermitian product. The Hermitian product of two complex vectors (thought of as $n$-by-$1$ matrices), say $v$ and $w$, is defined to be $\langle v,w\rangle = \overline{v}^{\top}\! w$.
Consider an $n$-by-$n$ matrix, say $M$, with complex entries acting on $\mathbb{C}^n$. The matrix preserves the Hermitian product if and only if $\langle Mv,Mw\rangle = \langle v,w\rangle$:
$$\langle Mv, Mw \rangle = \langle v,w\rangle \iff \overline{(Mv)}^{\top}(Mw) = \overline{v}^{\top}w \iff \overline{v}^{\top}(\overline{M}^{\top}\!\! M)w = \overline{v}^{\top}w \, .$$ Thus, $M$ preserves the Hermitian product if and only if $\overline{M}^{\top}\!\! M$ is the $n$-by-$n$ identity matrix, i.e. $M$ is a unitary matrix.
The unitary matrices are precisely those matrices which preserve the Hermitian inner product.
$$\langle v,w \rangle = \langle Uv,Uw \rangle$$
Hence it preserves the distance or length of a vector in the Unitary space (the finite-dimensional vector space over the complex number field $\mathbb{C}$ with an inner product) under rotation or reflection.