Optimizing quadratic form with respect to inner positive definite matrix with a trace constraint

Rather than solve for the Cholesky factor directly, find a solution in terms of a less structured matrix, $M$. Let a colon denote the matrix inner product, i.e. $$\eqalign{ A:B &= {\rm Tr}(AB^T) \cr M:M &= {\rm Tr}(MM^T) &= \frac{1}{\mu^2} \cr }$$ Also, for typing convenience let $$\eqalign{ A &= \frac{MM^T}{M:M},\quad\; Y &= Y^T = \sum_kz_kz_k^T - w_kw_k^T \cr }$$ Calculate the gradient of $F$. $$\eqalign{ F &= Y:A \cr dF &= Y:dA \cr &= Y:\Bigg(\frac{dM\,M^T+M\,dM^T}{M:M} - \frac{MM^T\big(dM:M+M:dM\big)}{(M:M)^2}\Bigg) \cr &= 2\mu^2\Big(YM-FM\Big):dM \cr \frac{\partial F}{\partial M} &= 2\mu^2\big(YM-FM\big) \cr }$$ Set the gradient to zero and solve for $M$. $$\eqalign{ &YM = F M \cr }$$ Thus it appears that the columns of $M$ are equal to the eigenvector $\{v_k\}$ of $Y$ associated with its minimum eigenvalue $\{\lambda_k\}$, i.e. $$k = \arg\min_j \lambda_j,\quad F = \lambda_k,\quad M = (\,v_k\;v_k\;v_k\;\ldots\,) \,=\, v_k{\large\tt 1}^T $$

Given the solution in terms of $M$, recover a solution in terms of $L$. $$\eqalign{ L &= {\rm cholesky}\bigg(\frac{MM^T}{M:M}\bigg) \cr A &= LL^T = \frac{MM^T}{M:M},\quad\quad {\rm Tr}(LL^T) = \frac{M:M}{M:M} = 1 \cr }$$

Define the $p\times n$ matrices $Z=[z_1,\dots,z_n]$ and $W=[w_1,\dots,w_n]$ (such that given vectors are respectively their columns). Convince yourself that you can rewrite your optimization problem as \begin{align} \min_{A} &<A,ZZ^T-WW^T> \\ &A\geq 0 ~,~<A,I> = 1 \end{align} where $A\geq 0$ implies $A$ should be positive-semi-definite. Also, for any two symmetric matrices $A,B$, we define $<A,B> = \mathrm{trace}(AB)$. We can define the eigen-decomposition \begin{align} A = \sum_{i=1}^{p}\lambda_iu_iu_i^T \end{align} where $u_i$ are the eigen vectors and $\lambda_i$ are the eigen values (of unit-norm). Let $B=ZZ^T-WW^T$. Convince yourselves that your optimization problem of finding $A$ is same as finding pairs $(\lambda_i,u_i)$ in the optimization problem \begin{align} \min_{\lambda_i,u_i}&\sum_{i=1}^{p}\lambda_iu_i^TBu_i \\& \lambda_i\geq 0~,~\forall i \\& \sum_{i=1}^{p}\lambda_i = 1 \end{align} From rayleigh-ritz ratio, it follows that for any unit-norm $u_i$, we have that \begin{align} u_i^TBu_i\geq \lambda_{min}(B) \end{align} and equality is achieved when $u_i$ is the eigen-vector corresponding to $\lambda_{min}(B)$. Thus, it follows that \begin{align}A = uu^T \end{align} where $u$ is the eigen-vector corresponding to smallest eigenvalue of $B$.