yet another determinant and inverse of a matrix

Based on the answer by მამუკა ჯიბლაძე, I can see a generalized claim for which we need a proof.

Define the polynomial $P_n(x)=\prod_{k=1}^n(c_k-x)$ and consider the matrices $$A_n:=\begin{bmatrix}c_1& a & a& \dots & a \\ b & c_2 & a& \ddots & a\\ b & b & c_3& \ddots & a \\ \vdots & \ddots & \ddots & \ddots & a \\ b & b & b & \dots & c_n \end{bmatrix}_{n\times n}.$$ Then, we have the determinantal evaluation $$\det(A_n)=\frac{a\cdot P_n(b)-b\cdot P_n(a)}{a-b}.\tag1$$ UPDATE. Here is a proof that I find. First, consider the matrices $$B_{n-1}:=\begin{bmatrix}a& a & a& \dots & a \\ c_2 & a & a& \ddots & a\\ b & c_3 & a& \ddots & a \\ \vdots & \ddots & \ddots & \ddots & a \\ b & b & b & \dots & a \end{bmatrix}_{n-1}, \qquad C_{n-1}:=\begin{bmatrix}b& c_2 & a& \dots & a \\ b & b & c_3& \ddots & a\\ b & b & b& \ddots & a \\ \vdots & \ddots & \ddots & \ddots & c_{n-1} \\ b & b & b & \dots & b \end{bmatrix}_{n-1}.$$ By subtracting the 2nd row (resp. column) from the 1st, and repeating this in $B_{n-1}$ (resp. $C_{n-1}$), it's easy to find that $$\det(B_{n-1})=a\prod_{k=2}^{n-1}(a-c_k) \qquad \text{and} \qquad \det(C_{n-1})=b\prod_{k=2}^{n-1}(b-c_k).$$ Next, consider the matrices $$E_{n-1}:=\begin{bmatrix}c_2& a & a& \dots & a \\ b & c_3 & a& \ddots & a\\ b & b & c_4& \ddots & a \\ \vdots & \ddots & \ddots & \ddots & a \\ b & b & b & \dots & c_n \end{bmatrix}_n, \qquad F_{n-2}:=\begin{bmatrix}c_2& a & a& \dots & a \\ b & c_3 & a& \ddots & a\\ b & b & c_4& \ddots & a \\ \vdots & \ddots & \ddots & \ddots & a \\ b & b & b & \dots & c_{n-1} \end{bmatrix}_{n-2}.$$ The determinants of $E_{n-1}$ and $F_{n-2}$ can be borrowed from the formula of that of $A_n$.

Finally, apply Dodgson's method of condensation and routinely (and inductively) verify that the formula in (1) yields the identity $$\det(F_{n-2})\cdot\det(A_n)=\det(A_{n-1})\cdot\det(E_{n-1})-\det(B_{n-1})\cdot\det(C_{n-1}).$$


Let's denote the vector $v=(1,1,\dots,1)$. Then we have that $$\det(A_n-av^{\intercal}v)=\prod_{k=1}^n(c_k-a) \quad \textrm{and}\quad\det(A_n-bv^{\intercal}v)=\prod_{k=1}^n(c_k-b)$$ since they are lower and upper triangular matrices, respectively. From the matrix determinant lemma we know that $$\det(A_n-av^{\intercal}v)=(1-avA_n^{-1}v^{\intercal})\det(A_n)\quad \textrm{and} \quad \det(A_n-bv^{\intercal}v)=(1-bvA_n^{-1}v^{\intercal})\det(A_n)$$ so combining everything together we get $$\frac{aP_n(b)-bP_n(a)}{a-b}=\frac{a\det(A_n-bv^{\intercal}v)-b\det(A_n-av^{\intercal}v)}{a-b}$$ $$=\frac{a(1-bvA_n^{-1}v^{\intercal})\det(A_n)-b(1-avA_n^{-1}v^{\intercal})\det(A_n)}{a-b}=\det(A_n)$$ which confirms your prediction.


Let me continue on from the notation and reasoning of Gjergji Zaimi's answer. By the Sherman-Morrison formula:

$$(A_n - a v^T v)^{-1} = A^{-1} + a \frac{ A^{-1} v v^T A^{-1} }{1 - a v^T A^{-1} v } =A^{-1} + a \frac{ A^{-1} v v^T A^{-1} }{\frac{ (a-b) P_n(a)}{a P_n(b) - bP_n(a)} } $$

$$(A_n - b v^T v)^{-1} = A^{-1} + b \frac{ A^{-1} v v^T A^{-1} }{1 - b v^T A^{-1} v } =A^{-1} + b \frac{ A^{-1} v v^T A^{-1} }{\frac{ (a-b) P_n(b)}{a P_n(b) - bP_n(a)} } $$

where in each case the denominator is calculated as a ratio of determinants using the determinant formulas in Gjergji Zaimi's answer.

Hence

$$b P_n(a) (A_n - a v^T v)^{-1} - aP_n(b) (A_n - b v^T v)^{-1} = (a P_n(b) - b P_n(A) ) A_n^{-1} + 0$$

as when we take a suitable linear combination, the second terms drop out, and

$$A_n^{-1} = \frac{ b P_n(a) (A_n - a v^T v)^{-1} - aP_n(b) (A_n - b v^T v)^{-1}}{ bP_n(a) - aP_n(b) }$$

Now $A_n - a v^T v$ and $A_n - b v^T v$, and their inverses, are lower-triangular and upper-triangular matrices respectively. I think these shouldn't be too hard to calculate explicitly.