No non-negative continuous function on $[a,b]$ such that $\int_a^b f(t)dt=1, \int_a^b tf(t)dt=c, \int_a^b t^2f(t)dt=c^2$ for $c\in\mathbb{R}$.

I'm assuming that $a \neq b$. Then, \begin{align*} |c| &= \left|\int_a^b tf(t) dt\right| \\ &= \left|\int_a^b t \sqrt{f(t)} \sqrt{f(t)} dt\right| \\ &\le \left(\int_a^b t^2 f(t) dt\right)^{1/2}\left(\int_a^b f(t) dt\right)^{1/2} \\ &= (c^2)^{1/2}\cdot 1^{1/2} \\ &= |c| \end{align*} Of course, the equality must hold, otherwise we have a contradiction. The equality condition in Cauchy-Schwarz is when $t \sqrt{f(t)} \propto \sqrt{f(t)}$, or $t \propto 1$, which is impossible for $t \in [a, b]$.

Hint: Check that $$ \langle u,v\rangle_f:=\int_a^b u(t)v(t)f(t)dt \quad \text{for all $u,v\in C([a,b])$}$$ defines an inner product. Then use that by assumption $$ \langle 1,t\rangle_f ^2 = \langle 1,1\rangle_f \langle t,t\rangle_f.$$ What do you know about the case in which equality holds in the Cauchy-Schwarz inequality?