Derivatives of eigenvalues

You need a convention for how to pick the magnitude of $v_p$; let's say it is normalized to unit length. Then the "standard" approach is to simply implicitly differentiate $Av_p = e_p v_p$: $$(dA)v_p + A(dv_p) = de_p \, v_p + e_p dv_p.$$

Now since $v_p$ is unit length, $v_p \cdot dv_p = \frac{1}{2} d(\|v_p\|^2) = 0$, so that $$v_p^T (dA) v_p + v_p ^T A \, dv_p = de_p + 0.$$ Finally since $A$ is Hermitian, $v_p^TA = e_p v_p^T$, and $$v_p^T (dA) v_p = de_p.$$

Of course, the surprisingly simple $\frac{\partial e_p}{\partial A_{ij}} = (v_p \cdot b_i)(v_p \cdot b_j)$ follows, where $b_i$ is the Euclidean basis vector with a $1$ at entry $i$ and zeroes elsewhere.

Now some caveats: if you dig into what's really going on in the above calculation, we're implicitly assuming that $e_p$ and $v_p$ vary smoothly given a variation of $A$. This is equivalent to the roots of the characteristic polynomial varying smoothly as a function of the coefficients, which is (only) true when the roots are distinct. As copper.hat alludes above, the situation is rather more complicated when $A$ has repeated eigenvalues. Moreover we've assumed explicitly that $A$ is Hermitian; if the variation you're interested in is the Hermitian $dA = b_ib_j^T + b_jb_i^T$, the above formula is different by a factor of two.

Side note: these functions come up in Hamiltonian geometry where they are coordinates of a moment map for a torus action on (a subset of) the Poisson manifold of Hermitian matrices (which can be identified with the dual Lie algebra of $U(n)$).

First do it for a diagonal hermitian matrix $D$ with distinct diagonal entries. There are two ways to move in the space of Hermitian matrices (i.e. the tangent space at a diagonal matrix decomposes as a sum of two subspaces):

• translate by another diagonal matrix $D_t = D+ tD'$. It's easy to see how this changes the eigenvalues.
• Act by conjugation by a unitary matrix: $A_t = e^{tX}De^{-tX}$. The eigenvalues are constant under this action, so the derivatives of the eigenvalues are zero in these directions.

Now since every Hermitian matrix can be diagonalized, you can use this to answer the question for all Hermitian matrices.

For degenerate eigenvalues, think of it this way. Either you have distinct eigenvalues $\lambda_1 > \ldots > \lambda_n$ or you have some equalities. Suppose we have a Hermitian matrix $A$ with some equalities, e.g. $\lambda_1 = \lambda_2 > \lambda_3$. The function $\lambda_1$ is not smooth on the set of Hermitian matrices at $A$. However, we can restrict to the subset of hermitian matrices whose eigenvalues have the same equalities (e.g. consider all hermitian matrices with eigenvalues $\lambda_1 = \lambda_2 > \lambda_3$, which is diffeomorphic to $\left(U(3)/U(2)\times U(1)\right)\times \mathbb{R}^2$). This subset is a smooth submanifold of the set of Hermitian matrices, so we can compute derivatives of functions on it.

The same trick as above can be applied again on this submanifold, with the exception that in the first part, when translating by another diagonal matrix, you can only translate by diagonal matrices that have the same "pattern" of equalities in their diagonal entries.