How do I compute the Singular Value Decomposition of the pseudo-inverse of a matrix?

Just work out the pseudo inverse of $S$. You can do this by working out the inverse of the square portion of $S$ and then padding the matrix with zeroes to obtain the appropriate size.

You first transpose $S$ then replace the non-zero entries with their reciprocals to get $S^{-1}$.

Alternatively, you can use the thin SVD where $S$ is diagonal, so $S^{-1}$ just reciprocates the non-zero entries of the diagonal of $S$.