Stein's lemma condition

Let $Z\sim \mathcal{N}(0,1)$ and $f$ a differentiable function with $E[|f'(Z)|]<\infty$. Then $$ \begin{align} E[Zf(Z)]&=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz=\int_{-\infty}^\infty \frac{1}{\sqrt{2\pi}}zf(z)\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz-f(0)E[Z]\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[f(z)-f(0)\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ &=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}z\left[\int_0^zf'(u)\,\mathrm du\right]\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz. \end{align} $$ On the other hand $$ \begin{align} E[f'(Z)]=&\int_{-\infty}^\infty f'(z)\frac{1}{\sqrt{2\pi}}\exp\left(-\frac{z^2}{2}\right)\,\mathrm dz\\ =&\frac{1}{\sqrt{2\pi}}\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz\\ +&\frac{1}{\sqrt{2\pi}}\int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz. \end{align} $$ So let us treat these two integrals seperately and use Fubini's theorem (justified by the assumption): $$ \int_0^\infty f'(z)\left[\int_z^\infty u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_0^\infty\int_z^\infty f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm du\,\mathrm dz\\ =\int_0^\infty \int_0^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du $$ and similarly $$ \begin{align} &\int_{-\infty}^0f'(z)\left[\int_{-\infty}^z-u\exp\left(-\frac{u^2}{2}\right)\mathrm du\right]\,\mathrm dz=\int_{-\infty}^0\int_{u}^0f'(z)(-u)\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du\\ &=\int_{-\infty}^0\int_{0}^u f'(z)u\exp\left(-\frac{u^2}{2}\right)\mathrm dz\,\mathrm du \end{align} $$ and thus $$ E[f'(Z)]=\int_{-\infty}^\infty\frac{1}{\sqrt{2\pi}}u\left[\int_{0}^u f'(z)\,\mathrm dz\right]\exp\left(-\frac{u^2}{2}\right)\,\mathrm du=E[Zf(Z)] $$

Given $E\left[ \lvert f'(z) \rvert \right] < \infty$, we want to show that $$\frac{1}{\sqrt{2\pi}} \left[ -f(z) \cdot \exp \left\{ \frac{-z^2}{2} \right\} \right] \Bigg|_{-\infty}^{\infty} = 0,$$ or alternatively that $$ \lim_{z \to \infty} \bigg\{ f(-z) \exp\left(-z^2 / 2\right) - f(z) \exp\left(-z^2 / 2\right) \bigg\} = \lim_{z \to \infty} \bigg\{ \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\} = 0. $$

Since $f$ is differentiable everywhere, we have that $f(z) - f(-z) = \int_{-z}^z f'(x) dx.$ Then $$ \begin{align} \bigg\lvert \frac{1}{\sqrt{2 \pi}} \left( f(-z) - f(z) \right) \exp\left(-z^2 / 2\right) \bigg\rvert &= \bigg\lvert \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) \int_{-z}^z f'(x) dx \bigg\rvert \\ &= \bigg\lvert \int_{-z}^z \frac{1}{\sqrt{2 \pi}} \exp\left(-z^2 / 2\right) f'(x) dx \bigg\rvert \\ &= \bigg\lvert \int_{-z}^z f'(x) \, \phi(z) \, dx \bigg\rvert \\ &\le \int_{-z}^z \big\lvert f'(x) \, \phi(z) \big\rvert \, dx \\ &= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(z) \, dx \\ &= \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(x)} \, dx \\ &\le \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \;\times\; \frac{\phi(z)}{\phi(0)} \, dx \\ &= \frac{\phi(z)}{\phi(0)} \int_{-z}^z \big\lvert f'(x) \big\rvert \, \phi(x) \, dx. \end{align} $$

The first factor, $\phi(z) / \phi(0)$, has limit 0. The second factor has limit $E\Big[\big\lvert f'(x) \big\rvert\Big]$, which is finite. So their product has limit 0, as desired.

I know I'm a few years late to the party, but I'm not sure about Dougal's and soren's solutions.

In Dougal's solution, the replacement of $\phi(x)$ by its maximum $\phi(0)$ in the denominator results in a smaller quantity rather than a larger one.

In soren's solution, I don't understand why $f'(Z)$ having finite expectation would imply that $f$ is Lipschitz. What if $f'(z) = z$ for instance? Then $f'(Z)$ has finite expectation but $f(z) = z^2/2$ isn't Lipschitz.

Stefan's solution seems fine, but I agree with gogurt that a more "direct" attack on the product term of the integration by parts might be informative.

Almost every source I could find takes the Casella and Berger approach, saying something like "it can be shown that the product term is zero." Or some of them give Stefan's proof. Finally, I found some course notes from a class taught by Sourav Chatterjee with a terse proof that the product term is zero. See Lemma 2.

Edit: The idea from Chatterjee's notes seems to be that you can show that $\mathbb{E}|f(Z)|$ is finite, which means that its integrand $f(z) \phi(z)$ must approach zero as $z \rightarrow \pm \infty$ if those limits exist (according to this).

He first shows that $\mathbb{E} |Z f(Z)|$ is finite: \begin{align*} \int_{-\infty}^\infty |zf(z)| \phi(z) dz &\leq \int_{-\infty}^\infty |z| \left[|f(0)| + |f(z) - f(0)|\right] \phi(z) dz\\ &\leq \int_0^\infty z \left[\int_0^z |f'(t)| dt\right] \phi(z) dz + \int_{-\infty}^0 (-z) \left[\int_z^0 |f'(t)| dt\right] \phi(z) dz + |f(0)| \sqrt{2/\pi}\\ &= \int_0^\infty |f'(t)| \underbrace{\int_t^\infty z \phi(z) dz}_{\phi(t)} dt + \int_{-\infty}^0 |f'(t)|\underbrace{ \int_{-\infty}^t (-z) \phi(z) dz}_{\phi(t)} dt + |f(0)| \sqrt{2/\pi}\\ &= \int_{\infty}^\infty |f'(t)| \phi(t) dt + |f(0)| \sqrt{2/\pi}\\ &= \mathbb{E} |f'(Z)| + |f(0)| \sqrt{2/\pi} \end{align*} Finally, take expectations of both sides of the pointwise inequality \begin{align*} |f(Z)| \leq \sup_{|t| \leq 1} |f(t)| + |Zf(Z)| \end{align*} The continuity of $f$ ensures that its supremum on $[-1, 1]$ is finite.