In trapezoid $ABCD$, $AB \parallel CD$ , $AB = 4$ cm and $CD = 10$ cm.

Let $OC = a$, $OD = b$. So $OA=\frac{2}{5}OC$, $OB = \frac{2}{5} OD$.

(Note you have swapped labels $C$ and $D$ in figure)

Also let $AD=3x$, $BC=3y$, so that $QA=2x$, $QB=2y$.

We have $a^2+b^2=100$

By Pythagoras, $$ (OA^2+OD^2)+(OB^2+OC^2)=9(x^2+y^2) $$ $$ \Rightarrow x^2+y^2=116/9 $$

By cosine-rule in $\triangle QAB$, $$ 4x^2+4y^2-4\sqrt{2}xy=4^2 $$ $$\Rightarrow xy=\dfrac{40\sqrt{2}}{9}$$

So $$ \begin{align} [ABCD] &= (1-\dfrac{4}{25})[QDC] \\ &=\dfrac{21}{25}(\frac{1}{2}\cdot QD\cdot QC\cdot\sin 45^{\circ})\\ &=\dfrac{21}{25}(\frac{1}{2}\cdot 5x\cdot5y\cdot\frac{1}{\sqrt{2}}) \\ &=\boxed{\dfrac{140}{3}} \end{align} $$

By your work $$(2x)^2+(2y)^2=4^2,$$ which gives $$x^2+y^2=4.$$ Also, $$AD=\sqrt{DO^2+AO^2}=\sqrt{25y^2+4x^2}$$ and $$BC=\sqrt{25x^2+4y^2}.$$ Now, let $PABC$ be parallelogram.

Thus, $P\in DC$, $AP=BC$, $$DP=DC-PC=10-4=6$$ and $$\measuredangle DAP=\measuredangle Q=45^{\circ}.$$ Thus, by the law of cosines for $\Delta DAP$ we obtain: $$\frac{25y^2+4x^2+25x^2+4y^2-36}{2\sqrt{(25x^2+4y^2)(25y^2+4x^2)}}=\frac{1}{\sqrt2}$$ or $$\frac{29\cdot4-36}{\sqrt2}=\sqrt{(25x^2+4y^2)(25y^2+4x^2)}$$ or $$3200=641x^2y^2+100(x^4+y^4)$$ or $$3200=441x^2y^2+100(x^2+y^2)^2,$$ which gives $$xy=\frac{40}{21}.$$ Id est, $$S_{ABCD}=\frac{1}{2}AC\cdot DB=\frac{1}{2}\cdot7x\cdot7y=\frac{140}{3}.$$