Prove that $\triangle ABC=\left(\triangle DEF \cdot \triangle XYZ\right)^{1/2}$

Without loss of generality, suppose that $[XYZ]$ (the area of $\triangle XYZ$) is $1$, and the ratio of similarity between $\triangle DEF$ and $\triangle XYZ$ is $r<1$ (so that $[DEF] = r^2$).

Let $a, b, c$ be the distances between $EF$ and $YZ$, between $ZX$ and $FD$, and between $XY$ and $DE$, respectively.

Then we have $[AEF] = \frac a2 \cdot EF$, $[BFD] = \frac b2 \cdot FD$, and $[CDE] = \frac c2 \cdot DE$ by the formula for triangle area; adding them together, we have $$[ABC] - [DEF] = \frac a2 \cdot EF + \frac b2 \cdot FD + \frac c2 \cdot DE.$$

On the other hand, we have $[AEY] = \frac a2 \cdot AY$, $[AFZ] = \frac a2 \cdot AZ$, $[BFZ] = \frac b2 \cdot BZ$, $[BDX] = \frac b2 \cdot BX$, $[CDX] = \frac c2 \cdot CX$, and $[CEY] = \frac c2 \cdot CY$; adding them together and noting that for example $YZ = AY + AZ$, we have $$[XYZ] - [ABC] = \frac a2 \cdot YZ + \frac b2 \cdot ZX + \frac c2 \cdot XY.$$

Because $r$ is the ratio of similarity between $\triangle DEF$ and $\triangle XYZ$, we have $EF = r \cdot YZ$, $FD = r \cdot ZX$, and $DE = r \cdot XY$, which tells us that
$$ [ABC] - [DEF] = r([XYZ] - [ABC]). $$ Recall that we assumed $[XYZ] = 1$ and $[DEF] = r^2$, so we now have $[ABC] - r^2 = r(1 - [ABC])$. Solving, we get $[ABC] = r$, so $[ABC] = \sqrt{r^2 \cdot 1} = \sqrt{[DEF] \cdot [XYZ]}$.

Say, $\triangle DEF = p$, then triangle $\triangle XYZ = p(t^2)$ where t is the ratio of sides of $\triangle XYZ$ to $\triangle ABC$.

$\triangle XYZ = [XDEY] + [YEFZ] + [XDFZ] + \triangle DEF$ (3 parallelograms + $\triangle DEF$).

Say, $EF = a, FD = b, DE = c$

$\triangle XYZ = \dfrac{1}{2}[c(1+t)h_3 + a(1+t)h_1 + b(1+t)h_2] + \triangle DEF$ $p(t^2) = \dfrac{1}{2}[c(1+t)h_3 + a(1+t)h_1 + b(1+t)h_2] + p$

$2p(t^2) = c(1+t)h_3 + a(1+t)h_1 + b(1+t)h_2 + 2p$ ...(i)

Now, $\triangle ABC = \triangle CDE + \triangle AEF + \triangle BDF + \triangle DEF$

$\triangle ABC = \dfrac{1}{2}(c.h_3 + a.h_2 + b.h_1) + p$ ...(ii)

From (i) and (ii),

$p(t^2) = (\triangle ABC - p)(1+t) + p$

$p(t-1) = \triangle ABC - p$

$\triangle ABC = pt = \sqrt{p.pt^2} = \sqrt{\triangle DEF.\triangle XYZ}$