Paradox on the derivative of the rank of a matrix?

Your assumption that $f$ has no derivative anywhere is wrong.

In fact, the derivative exists and is $0$ almost everywhere.

Every matrix with full rank has a neighborhood of matrices that also have full rank, so in this neighborhood $f$ is constant and thus differentiable.

For matrices $X$ that do not have full rank, $f$ is not differentiable at $X$. (The pseudoinverse is not differentiable at such points either; it isn't even continuous).