A limit like a Riemann sum

Here, we sketch a way forward. First, we assume that $f$ is strictly positive.

Then, we can write

$$\sqrt[n]{f(k/n)}=e^{\frac1n \log(f(k/n))}=1+\frac{\log(f(k/n))}{n}+O\left(\frac1{n^2}\right) \tag1$$

Using $(1)$ it is easy to see that

$$\begin{align} \left(\frac1n\sum_{k=1}^n\sqrt[n]{f(k/n)}\right)^n&=\left(\frac1n\sum_{k=1}^n\left(1+\frac{\log(f(k/n))}{n}+O\left(\frac1{n^2}\right)\right)\right)^n\\\\ &=\left(1+\frac{\frac1n \sum_{k=1}^n\log(f(k/n))}{n}+O\left(\frac1{n^2}\right)\right)^n \end{align}$$

whereupon taking the limit yields

$$\lim_{n\to \infty}\left(\frac1n\sum_{k=1}^n\sqrt[n]{f(k/n)}\right)^n=e^{\int_0^1\log(f(x))\,dx}$$

@Dr.MV has sketched a nice proof that

$$\tag 1 U_n(f) \to \exp (\int_0^1 \ln f(x)\, dx)$$

if $f$ is positive and continuous on $[0,1].$ But what if positivity is relaxed to nonnegavity, as in the statement of the problem? In that case we could have $\int_0^1 \ln f = -\infty.$ But $(1)$ might still be true if we use the natural convention $e^{-\infty} = 0.$ For example if $f=0$ on any subinterval, then $(1)$ still holds. It also holds for $f(x) = e^{-1/x}.$

Interestingly, $(1)$ can fail for $f$ nonnegative, even if $\int_0^1 \ln f$ converges. To get the gist of the idea, let's first find a Riemann integrable, though not continuous, function for which the result fails. Choose a sequence of positive integers $1<m_1 < m_2 < \cdots,$ with $m_{k+1} > m_k^2$ for all $k.$ For each $k=1,2,\dots ,$ consider the finite sets

$$F_k = \{1/m_k^2, 2/m_k^2,3/m_k^2, \dots ,m_k/m_k^2 = 1/m_k\}.$$

The $F_k$ are pairwise disjoint, and every point in $F_{k+1}$ lies to the left of every point in $F_k.$

Define

$$f(x) = \begin{cases} 0,& x\in \cup_{k=1}^\infty F_k\\x, & x\in [0,1]\setminus \cup_{k=1}^\infty F_k\end{cases}$$

Then $f$ is Riemann integrable on $[0,1],$ and $\int_0^1\ln f(x)\,dx = \int_0^1\ln x\,dx = -1.$ But note that for each $k,$

$$\frac{1}{m_k^2} \sum_{j=1}^{m_k^2}f(j/m_k^2)^{1/m_k^2} = \frac{1}{m_k^2}\sum_{j=m_k+1}^{m_k^2}f(j/m_k^2)^{1/m_k^2} \le \frac{1}{m_k^2}\sum_{j=m_k+1}^{m_k^2}1$$ $$ = \frac{1}{m_k^2}(m_k^2-m_k) = 1- 1/m_k.$$

Thus as $k\to \infty,$ $U_{m_k^2} \le (1- 1/m_k)^{m_k^2} \to 0.$ Therefore $(1)$ fails for this $f$ as claimed.

How do we show there is a continuous counterexample? We model it on the the above. At this point I'll be descriptive rather than precise. From each point $ a\in\cup_{k=1}^\infty F_k,$ we consider two nearly vertical rays from $(a,0)$ of positive and negative slope. These rays will intersect $y=x$ in points $(b,b),(c,c),$ where $b<a<c.$ We can make $c-b$ as small as we like. The definition of $f$ on $[b,c]$ is then the piecwise linear function connecting $(b,b),(a.0),(c,c).$ If we do this methodically for every point in $\cup_{k=1}^\infty F_k,$ making each "$[b,c]$ interval" very small, and if we define $f(x)=x$ everywhere else, we'll have a continuous counterexample.