Is it possible to generalize this mean value theorem for integrals?

Right, $\varphi$ need not be continuous.

But it must be such that the two integrals exist. (So it is enough that $\varphi$ be Lebesgue integrable. But probably you learn this theorem before you learn about the Lebesgue integral.)

However, $f$ must be continuous to let us use the intermediate value theorem.

The continuity is used to prove that if $\phi(x)>0$ for some $x$ then exists a neighborhood $J:=(x-t,x+t)$ of $x$ such that $\phi(x)>0,\forall x \in J$

So the integral will be positive,since $\phi$ is non-negative in general.

If $\phi$ is zero,then this case is covered in the proof.

If $\phi$ is not everywhere continuous then there are two cases:

If $\phi$ is positive for only finite $x$'s and $\phi$ is not continuous at that points,then this case is covered as the case where $\phi=0$

If $\phi$ is positive gain for finite $x$'s and is continuous at one such point,then will exists a neighborhood again where $\phi$ is poisitive,so the same argument is applied.

As it is mentioned in the comments and in the other answer,$f$ must be continuous.