L2 Norm of Pseudo-Inverse Relation with Minimum Singular Value
Hint: It suffices to prove the following facts:
- If $U,V$ are orthogonal (and square), then $\|AV\| = \|UA\| = \|A\|$
- $(U\Sigma V^T)^\dagger = V\Sigma^{\dagger}U^T$ (where $\dagger$ denotes the pseudo-inverse)
- If $\Sigma$ is a diagonal matrix of singular values, then $\|\Sigma^{\dagger}\| = 1/\sigma_{min}(\Sigma)$
Perhaps you can put the pieces together from here. Happily, this approach still works when $A$ does not have full column-rank.