Projection of a vector onto the null space of a matrix

It might be easier to use some (related) facts from linear algebra. Implicit in Bersekas' solution is the fact that $A$ has full rank, which is equivalent to $A A^*$ being invertible.

The space $\ker A$ is a (closed) subspace, and the problem is to find the nearest point in the subspace to the point $z$. It is straightforward (using a compactness argument) to show that a solution exists.

In the following, I am assuming that $A$ has full rank.

At a solution $\hat{x}$, we have $\|z-x\|^2 \ge \|z-\hat{x}\|$ for all $x \in \ker A$. Writing $\|z-x\|^2 = \|z-\hat{x}\|^2 + \|x-\hat{x}\|^2 - 2 \operatorname{re} \langle z-\hat{x}, x-\hat{x}\rangle $ and combining gives $\|x-\hat{x}\|^2 \ge 2 \operatorname{re} \langle z-\hat{x}, x-\hat{x}\rangle $, and since $\ker A$ is a subspace, this shows that $z-\hat{x} \bot \ker A$.

Since $\ker A = ({\cal R}A^T)^\bot$, we can write $z-\hat{x} = A^* \hat{y}$ for some $\hat{y}$, and since $\hat{x} \in \ker A$, we have $A z= A A ^* \hat{y}$ and since we have assumed full rank, we have $ \hat{y} = (A A ^*)^{-1} A z$, and so the solution is given by $\hat{x} = z-A^* \hat{y} = (I-A^*(A A ^*)^{-1} A)z $.