Find the length of a diagonal in an inscribed (non-regular) hexagon

Ptolemy's theorem helps.

Let $|\overline{BE}|=x$.

Applying the theorem to the quadrilateral $CDEB$, we get $$|\overline{CD}|\cdot|\overline{BE}|+|\overline{DE}|\cdot|\overline{CB}|=|\overline{BD}|\cdot|\overline{CE}|$$ Since $|\overline{BD}|=|\overline{CE}|$, we get $$|\overline{BD}|=|\overline{CE}|=\sqrt{5x+4}$$ Since $|\overline{DF}|=|\overline{CE}|$, we get $$|\overline{DF}|=\sqrt{5x+4}\tag1$$

Applying the theorem to the quadrilateral $CDEF$, we get $$|\overline{CD}|\cdot|\overline{EF}|+|\overline{DE}|\cdot|\overline{CF}|=|\overline{CE}|\cdot|\overline{DF}|$$ so $$|\overline{CF}|=\frac{5x-21}{2}$$ Since $|\overline{AD}|=|\overline{CF}|$, we get $$|\overline{AD}|=\frac{5x-21}{2}\tag2$$ where we have to have $\frac{5x-21}{2}\gt 0,$ i.e. $$x\gt\frac{21}{5}\tag3$$

Applying the theorem to the quadrilateral $CDFA$, we get $$|\overline{CD}|\cdot|\overline{AF}|+|\overline{AC}|\cdot|\overline{DF}|=|\overline{AD}|\cdot|\overline{CF}|$$ Since $|\overline{AC}|=|\overline{DF}|$, we get $$55+|\overline{DF}|^2=|\overline{AD}|^2\tag4$$

From $(1)(2)(4)$, we have $$55+(5x+4)=\left(\frac{5x-21}{2}\right)^2$$ i.e. $$(x-1)(5x-41)=0$$ It follows from $(3)$ that $$|\overline{BE}|=x=\frac{41}{5}$$

I'll leave a couple of observations below. Neither qualifies as "smarter", but may help someone find what must be the trick solution.

1.   The problem is overdetermined, in the sense that the figure can be solved without knowing the area. For example, let $r$ be the radius, and $2a,2b$ the center angles subtended by $AB,BC\,$, then:

$$ 2r\sin(a)=5 \\ 2r \sin(b) = 2 \\ 2r \sin(3a+2b)=11 $$

The system can technically be solved for $r, a, b\,$, and the rest follows.

2.   Let $BE=x\,$, then adding the areas of the two trapezoids gives:

$$ \frac{1}{2}\left(x+11\right)\sqrt{5^2 - \frac{1}{4}(11-x)^2} + \frac{1}{2}\left(x+5\right)\sqrt{2^2 - \frac{1}{4}(x-5)^2}= 54 $$

After not so pretty calculations, the above reduces to a quartic equation:

$$ \begin{align} 0 &\;=\; 1825 x^4 + 7820 x^3 - 244362 x^2 - 1594900 x + 16946161 \\ &\;=\; (5 x - 41)^2 \,(73 x^2 + 1510 x + 10081) \end{align} $$

The unique real root is obviously the answer, but what's more interesting is that it's a double root. This would seem to suggest that the chosen numbers represent some sort of "limit case", and once it is recognized what is so special about it, that would likely point to the better solution.