Functional logarithmic equation
No nontrivial solutions exist.
Note that since $\ln$ is a concave function, we have for any $c \in \mathbb{R}^+$ \begin{align} \ln(x) \leq \frac{1}{c}(x-c) + \ln(c) = \frac{x}{c} - 1 + \ln(c) \end{align} Therefore, we get \begin{align} \int_1^2 \ln(f(x))dx \leq& \int_1^2 \frac{f(x)}{c}-1+\ln(c) dx \\ =& \frac{1}{c}\int_1^2 f(x)dx -1 + \ln(c) \end{align} Substituting $c = \int_1^2 f(x)dx$, we find that \begin{align} \int_1^2 \ln(f(x))dx \leq \ln \left( \int_1^2 f(x)dx \right) \end{align} Hence, if $f$ solves the equation, we must have for any $h: \mathbb{R} \to \mathbb{R}$ that \begin{align} \left. \frac{d}{dt} \right|_{t=0} \left( \ln \left( \int_1^2 f(x)+t \cdot h(x) dx \right) - \int_1^2 \ln(f(x) + t \cdot h(x))dx \right) = 0 \end{align} Simplifying this, we get that if $f$ is a solution, we must have for any $h: \mathbb{R} \to \mathbb{R}$ \begin{align} \frac{\int_1^2 h(x)dx}{\int_1^2 f(x)dx} = \int_1^2 \frac{h(x)}{f(x)}dx \end{align} This is clearly a quite strong condition on $f$. It is equivalent to saying that \begin{align} \int_1^2 h(x) \left(1 - \frac{\int_1^2 f(t)dt}{f(x)} \right)dx = 0 \end{align} for all $h$. In particular, it must hold for $h = \left(1 - \frac{\int_1^2 f(t)dt}{f(x)} \right)$. From this we find \begin{align} \int_1^2 \left(1 - \frac{\int_1^2 f(t)dt}{f(x)} \right)^2 dx =& 0 \\ \left(1 - \frac{\int_1^2 f(t)dt}{f(x)} \right) =& 0 \\ f(x) =& \int_1^2 f(t)dt \end{align} These last two equations must hold almost everywhere on $[1,2]$. Hence, $f$ must be constant almost everywhere on $[1,2]$. The condition of differentiability of $f$ was not necessary.