Proof of relationship $S^2−S(a+b+c+d+e)+ab+bc+cd+de+ea=0$ between areas connected to a pentagon

Here is a proof using barycentric coordinates.

Let us use compact notation $[MNP]$ for the oriented area of triangle $MNP$ (positive is $M,N,P$ in this order has the direct orientation, negative otherwise).

enter image description here

Let us consider $ABC$ as the reference triangle.

The barycentric coordinates (b.c. in short) of points $D$ and $E$ are

$$b.c.(D)=(\underbrace{\dfrac{[\color{red}{D}BC]}{[ABC]}}_{(+)},\underbrace{\dfrac{[A\color{red}{D}C]}{[ABC]}}_{(-)},\underbrace{\dfrac{[AB\color{red}{D}]}{[ABC]}}_{(+)})=(\dfrac{b}{a},\dfrac{a+d-S}{a},\dfrac{S-b-d}{a})$$

(notations (+), resp. (-), for positive, resp. negative orientation) and:

$$b.c.(E)=(\dfrac{[\color{red}{E}BC]}{[ABC]},\dfrac{[A\color{red}{E}C]}{[ABC]},\dfrac{[AB\color{red}{E}]}{[ABC]})=(\dfrac{S-c-e}{a},\dfrac{a+c-S}{a},\dfrac{e}{a}).$$

Let us consider as well $A$ with $b.c.(A)=(1,0,0)$ evidently.

Let us now express a classical property: the determinant of the barycentrical coordinates of 3 points $M,N,P$ is equal to the ratio $[MNP]/[ABC]$ of the area of the triangle to the area of the reference triangle giving here for triangle $MNP \equiv ADE$:

$$\begin{vmatrix}1&b/a&(S-c-e)/a\\0&(a+d-S)/a&(a+c-S)/a\\0&(S-b-d)/a&e/a\end{vmatrix}=\dfrac{d}{a}$$

factorizing $1/a$ in the second and third columns, we obtain the equivalent form:

$$\begin{vmatrix}(a+d-S)&(a+c-S)\\(S-b-d)&e\end{vmatrix}=ad \tag{1}$$

Expanding (1) gives the looked for relationship:

$$S^2−S(a+b+c+d+e)+(ab+bc+cd+de+ea)=0\tag{2}$$

Important edit: I have finally found a very interesting reference to formula (2) in [this answer] (https://mathoverflow.net/q/151316) [and also used here which is another answer to the same question]. More than that, the author of the answer establishes a "mirror formula", surprisingly, identical to (2):

$$S^2−S(a'+b'+c'+d'+e')+(a'b'+b'c'+c'd'+d'e'+e'a')=0\tag{3}$$

with the complementary areas:

$$a':=S-a, \ b':=S-b, \, c':=S-c, \ d':=S-d, \ e':=S-e$$

i.e. areas of quadrilaterals $CDEA, DEAB,....$ !

Remark 1: It is explained in the given reference that by solving (2) as a quadratic one obtains a formula for $S$ as a function of $a,b,c,d,e$ under the condition to take the largest root, whereas, doing the same in (3), one has to take the smallest root.

Remark 2: There is a connection of (2) with the shoelace formula.


Let $AC\cap BE=\{K\}$, $AC\cap BD=\{L\}$, $CE\cap BD=\{M\}$, $CE\cap AD=\{N\}$ and $AD\cap BE=\{P\}$.

Since $$S_{\Delta BLC}\cdot S_{\Delta ALD}=S_{\Delta ABL}\cdot S_{\Delta CLD},$$ we obtain $$S_{\Delta BLC}\left(S-a-b-d+S_{\Delta BLC}\right)=\left(a-S_{\Delta BLC}\right)\left(b-S_{\Delta BLC}\right),$$ which gives $$S_{\Delta BLC}=\frac{ab}{S-d}.$$ Similarly we can get $$S_{\Delta CMD}=\frac{bc}{S-e},$$ $$S_{\Delta DNE}=\frac{cd}{S-a},$$ $$S_{\Delta APE}=\frac{de}{S-b}$$ and $$S_{\Delta ABK}=\frac{ea}{S-c}.$$ Id est, $$\frac{e}{S-c}\cdot\frac{e}{S-b}=\frac{AK}{AC}\cdot\frac{AP}{AD}=\frac{S_{\Delta}AKP}{S_{\Delta}ACD}=\frac{e-\frac{ea}{S-c}-\frac{de}{S-b}}{S-a-d},$$ which gives $$S^2-(a+b+c+d+e)S+ab+bc+cd+de+ea=0.$$