Does $\det(A+xB)$ have a nice form for $3 \times 3$ matrices?

First notice that setting $x = 0$ gives that the constant term in $x$ is $\det A$, and appealing to symmetry gives that the leading term is $(\det B) x^3$, so it remains to find the two intermediate terms.

Temporarily assume $A$ is invertible---we'll remove this restriction later. Factoring gives $A + x B = A (I + x A^{-1} B),$ and taking the determinant gives \begin{align} \det(A + x B) &= \det A \det (I + x A^{-1} B) \\ &= \det A [1 + x \operatorname{tr}(A^{-1} B) + x^2 \sigma_2(A^{-1} B) + x^3 \det (A^{-1} B)] . \end{align} Here we've just exploited the definition of eigenvalues to write $\det (I + C)$ in terms of the usual symmetric polynomials $\operatorname{tr}, \sigma_2, \det$ of the eigenvalues of $\lambda_i$. Explicitly, $$\sigma_2(C) = \lambda_2 \lambda_3 + \lambda_3 \lambda_1 + \lambda_1 \lambda_2 = (c_{22} c_{33} - c_{23} c_{32}) + (c_{33} c_{11} - c_{31} c_{13}) + (c_{11} c_{22} - c_{12} c_{21}) .$$

So, the coefficient of $x$ in $\det (A + x B)$ is $$\det A \operatorname{tr}(A^{-1} B) .$$ We can write this without using the inversion operator, after which we can appeal to continuity to show that our resulting formula doesn't require the condition that $A$ is invertible.

Using the same trick as before lets us expand the characteristic polynomial of $A$ in terms of symmetric polynomials of the eigenvalues of $A$: $$\phantom{(\ast)} \qquad p_A(t) = \det(t I - A) = t^3 - (\operatorname{tr} A) t^2 + \sigma_2(A) t - \det A . \qquad (\ast)$$ In particular, the Cayley-Hamilton Theorem gives $$0 = p_A(A) = A^3 - (\operatorname{tr A}) A^2 + \sigma_2(A) A - (\det A) I ,$$ and (using again that $A$ is invertible) rearranging gives $$A^{-1} = (\det A)^{-1} [A^2 - (\operatorname{tr} A) A + \sigma_2(A) I].$$ Substituting now gives that the coefficient of $x$ in $\det(A + x B)$ is \begin{align} \det A \operatorname{tr}(A^{-1} B) &= \det A \operatorname{tr}[(\det A)^{-1} [A^2 - (\operatorname{tr} A) A + \sigma_2(A) I] B] \\ &= \boxed{\operatorname{tr}(A^2 B) - \operatorname{tr}(AB) \operatorname{tr} A + \sigma_2(A) \operatorname{tr} B} . \end{align} The coefficient of $x$ and the boxed expression are continuous functions that agree where $A$ is invertible, and since this is dense in the set of all $3 \times 3$ matrices, continuity implies that they agree everywhere.

We can calculate the coefficient of $x^2$ in $\det(A + x B)$ similarly, but, as we did for the leading and constant terms in $x$ we can just appeal to symmetry, giving $$\boxed{\operatorname{tr}(B^2 A) - \operatorname{tr}(BA) \operatorname{tr} B + \sigma_2(B) \operatorname{tr} A} .$$ Putting this all together and rearranging a little for aesthetic reasons gives $$\color{#df0000}{\boxed{\det (A + x B) = x^3 \det B + [\operatorname{tr}(A B^2) - \operatorname{tr} B \operatorname{tr}(AB) + (\operatorname{tr} A) \sigma_2(B)] x^2 \\ \qquad\qquad\qquad\qquad\qquad\qquad + [\operatorname{tr}(A^2 B) - \operatorname{tr} A \operatorname{tr}(AB) + \sigma_2(A) \operatorname{tr} B] x + \det A}} .$$

Finally, we have $$\sigma_2(C) = \lambda_2 \lambda_3 + \lambda_3 \lambda_1 + \lambda_1 \lambda_2 = \frac{1}{2}[(\operatorname{tr} C)^2 - \operatorname{tr} (C^2)],$$ so substituting lets us produce an expression that relies only on the more familiar trace and determinant operators:

$$\color{#df0000}{\boxed{\det (A + x B) = x^3 \det B + \left[\operatorname{tr}(A B^2) - \operatorname{tr} B \operatorname{tr}(AB) + \frac{1}{2} (\operatorname{tr} A) [(\operatorname{tr} B)^2 - \operatorname{tr} (B^2)]\right] x^2 \\ \qquad\qquad\qquad\qquad\qquad\qquad + \left[\operatorname{tr}(A^2 B) - \operatorname{tr} A \operatorname{tr}(AB) + \frac{1}{2}[(\operatorname{tr} A)^2 - \operatorname{tr} (A^2)] \operatorname{tr} B\right] x + \det A}} .$$

The general procedure here works just as well for finding the coefficients of $\det(A + x B)$ for matrices of any size.


Let $A_1,A_2,A_3$ the columns of $A\in M_3(\mathbb{C})$ and $B_1,B_2,B_3$ the columns of $B\in M_3(\mathbb{C})$. For the property of the determinant of a matrix to be a multilinear application of its columns we must \begin{align} \det(A+xB) =& \det \Big(A_1+xB_1\;\Big|\;A_2+xB_2\;\Big|\; A_3+xB_3 \Big) \\ =& \det \Big(A_1\;\Big|\;A_2+xB_2\;\Big|\; A_3+xB_3 \Big) \\ +&\det \Big(xB_1\;\Big|\;A_2+xB_2\;\Big|\; A_3+xB_3 \Big) \\ =& \det \Big(A_1\;\Big|\;A_2\;\Big|\; A_3+xB_3 \Big) \\ +& \det \Big(A_1\;\Big|\; xB_2\;\Big|\; A_3+xB_3 \Big) \\ +& \det \Big(xB_1\;\Big|\;A_2\;\Big|\; A_3+xB_3 \Big) \\ +& \det \Big(xB_1\;\Big|+xB_2\;\Big|\; A_3+xB_3 \Big) \\ =& \det \Big(A_1\;\Big|\;A_2\;\Big|\; A_3 \Big) \\ +& \det \Big(A_1\;\Big|\;A_2\;\Big|\;xB_3 \Big) \\ +& \det \Big(A_1\;\Big|\; xB_2\;\Big|\; A_3\Big) \\ +& \det \Big(A_1\;\Big|\; xB_2\;\Big|\; xB_3 \Big) \\ +& \det \Big(xB_1\;\Big|\;A_2\;\Big|\; A_3 \Big) \\ +& \det \Big(xB_1\;\Big|\;A_2\;\Big|\; xB_3 \Big) \\ +& \det \Big(xB_1\;\Big|\;xB_2\;\Big|\; A_3\Big) \\ +& \det \Big(xB_1\;\Big|\;xB_2\;\Big|\; xB_3 \Big) \end{align} Then \begin{align} \det(A+xB) =& \det(A) \\ &+ x\left(\det(A_1|A_2|B_3)+\det(A_1|B_2|A_3)+\det(B_1|A_2|B_3)\right) \\ &+ x^2\left(\det(A_1|B_2|B_3)+\det(B_1|A_2|B_3)+\det(B_1|B_2|A_3)\right) \\ &+ x^3\det(B) \end{align}