On a monotonicity property of Fourier coefficients of truncated power functions

I prefer to write $2\pi$ in the exponent and reverse the direction of the monotonicity (none of which really matters, of course), so we shall fix $k>1$ (real in general), $n\in\mathbb Z_+$ (that is essential) and define $$ a_n=\Re I_n, \qquad I_n=\int_0^1 (1-z)^ke^{2\pi in z}dz $$ (I immediately write the integral in the complex notation; the branch of the $k$-th power is positive on $(0,1)$; all our paths will be in the upper half-strip $0\le x=\Re z\le 1, y=\Im z\ge 0$).

We want to show that $a_n>0$ and $a_{n+1}<a_n$. There is nothing to do for $n=0$, so from now on, we assume that $n\ge 1$.

Two (possibly overlapping) cases are possible: $k>4n$ and $k-1<4n$. We shall use different contours for these two cases.

Case 1: $k>4n$. Let $\rho=\frac {4n}{k}<1$. Consider the contour given by the equation $-k\alpha+2\pi n x=0$ (or, $x=\frac{2\alpha}{\pi\rho}$, $0\le\alpha\le\rho\frac{\pi} 2$) where the notation is clear from the following picture:

The explicit equation for the curve (using $\alpha$ as the parameter) is $$ x=\frac{2\alpha}{\pi\rho},\quad y=\left(1-\frac{2\alpha}{\pi\rho}\right)\tan\alpha\,. $$ Differentiating, we get $\dot x=\frac{2}{\pi\rho}$, $\dot y=\left(1-\frac{2\alpha}{\pi\rho}\right)\frac 1{\cos^2\alpha}-\frac{2}{\pi\rho}\tan\alpha$, so for the direction $\theta$ of $dz$, we get $$ \tan\theta=\left(\frac{\pi\rho}2-\alpha\right)\frac 1{\cos^2\alpha}-\tan\alpha\,. $$ At the very least, we see that $-\alpha\le\theta\le\frac{\pi}2$.

Now let $F_n(z)$ be the integrand in $I_n$. We have $F_{n+1}(z)=e^{2\pi iz}F_n(z)$, so $|F_{n+1}(z)|=e^{-2\pi y}|F_n(z)|<|F_n(z)|$ while $\arg F_{n+1}(z)=\arg F_{n}(z)+2\pi x=\arg F_{n}(z)+\frac{4\alpha}\rho$ on our contour. Note that $\arg F_{n}(z)=-k\alpha+2\pi n x=0$ on our contour, so we have $$ a_n=\int |F_n|\cos\theta|dz| $$ and $$ a_{n+1}=\int |F_{n+1}|\cos(\theta+\tfrac {4\alpha}\rho)|dz|\,. $$ Since $\cos\theta>0$ and $|F_{n+1}|<|F_{n}|$, all we need now is to show that $$ \cos(\theta+\tfrac {4\alpha}\rho)\le \cos\theta $$ If $-\alpha\le\theta\le 0$, we rotate by $\frac{4\alpha}\rho\in (4\alpha,2\pi)$, so we go beyond the symmetric angle $-\theta$ but stay short of the full revolution, so the inequality certainly holds. Now we need to consider $\theta>0$ only, in which case we want to show that we cannot rotate beyond $2\pi-\theta$, i.e., that $\theta+\frac{2\alpha}\rho\le\pi$ or $$ \theta\le \pi-\beta, \beta=\frac{2\alpha}\rho\,. $$ Note that $0\le\beta<\pi$ and if $\beta\le\frac\pi 2$, then there is nothing to prove because $\theta\le\frac\pi 2$.

So, assume $\beta>\frac \pi 2$. Then we need $\tan\theta\le\tan(\pi-\beta)$, i.e. (rewriting $\alpha$ in terms of $\beta$), $$ -\tan(\tfrac\rho 2\beta)+\frac\rho 2(\pi-\beta)\frac 1{\cos^2\tfrac\rho 2\beta}\le\tan(\pi-\beta). $$ Bringing the terms on the left hand side to the common denominator, we get $$ LHS=\frac{\rho (\pi-\beta)-\sin(\rho\beta)}{2\cos^2\tfrac\rho 2\beta}\le \rho \frac{(\pi-\beta)-\sin(\beta)}{2\cos^2\tfrac 1 2\beta} \\ \le \frac{(\pi-\beta)-\sin(\beta)}{1+\cos\beta}= \frac{(\pi-\beta)-\sin(\pi-\beta)}{1-\cos(\pi-\beta)} $$ because $\rho<1$, $\sin(\rho\beta)\ge \rho\sin\beta$ and $\cos \tfrac\rho 2\beta\ge \cos \tfrac 1 2\beta$.

Let now $\tau=\pi-\beta\in(0,\frac\pi 2)$. Our inequality becomes $$ \frac{\tau-\sin\tau}{1-\cos\tau}\le\frac{\sin\tau}{\cos\tau}\,, $$ that is $$ \tau\cos\tau\le\sin\tau\,, $$ which is classical.

This finishes Case 1.

Case 2: $k-1<4n$. This case is way more interesting. We will not be able here to join $0$ to $1$ directly and will make a detour via $+i\infty$. In this case we define $\rho=\frac{4n}{k-1}>1$ and use the same equation but with $0\le\alpha<\frac \pi 2$. Even as $\alpha\to\frac\pi 2-$, we shall have only $x\to \frac 1\rho<1$ while $y\to+\infty$ with the picture like this:

Of course, once we escape to $+i\infty$, we have to come back from it, which we will do over the right boundary line $z=1+it$ of the strip. That the abscissa of the escape is different from that of the return is not a problem because $F_n,F_{n+1}$ decay to $0$ when we go up, so we may think that we join the paths high up by a horizontal interval and then take the limit.

We still have $\theta<\frac\pi 2$. I claim that now $\theta>0$. Indeed, $\tan\theta$ is given by the same formula (only now $\rho>1$!), so we have $$ \tan\theta=\left(\frac{\pi\rho}2-\alpha\right)\frac 1{\cos^2\alpha}-\tan\alpha\ge \left(\frac{\pi}2-\alpha\right)\frac 1{\cos^2\alpha}-\tan\alpha \\ =\frac{(\pi-2\alpha)-\sin(2\alpha)}{2\cos^2\alpha} =\frac{(\pi-2\alpha)-\sin(\pi-2\alpha)}{2\cos^2\alpha}>0\,. $$ Note that now the argument regime of $F_n(z)dz$ is $\theta-\alpha$ because $-(k-1)\alpha$ cancels with $2\pi nx$. Thus, on the way up, we have $$ \int |F_n(z)|\cos(\theta-\alpha)|dz| $$ we look at the way down now matching to $z$ the point $z'=1+i\Im z$ (which is a legitimate parameterization). Notice that $|F_n(z')|=\sin^k\alpha|F_n(z)|$ and the argument of $F_n(z')dz'$ is constant and does not depend on $n$ (only the power factor matters) and $|dz'|=\sin\theta|dz|$. Thus, if we combine both paths, we shall get $$ \int_{\text{upward path}} |F_n(z)|[\cos(\theta-\alpha)-\lambda\sin\theta\sin^k\alpha]|dz| $$ where $\lambda\in[-1,1]$ is some constant (common for $n$ and $n+1$). The key point now is that $$ \cos(\theta-\alpha)=\cos\theta\cos\alpha+\sin\theta\sin\alpha \ge\sin\theta\sin\alpha\ge |\lambda\sin\theta\sin^k\alpha| $$ so the factor is, again, positive.

For $F_{n+1}$, we get $$ \int_{\text{upward path}} |F_{n+1}(z)|[\cos(\theta-\alpha+\tfrac{4\alpha}{\rho})-\lambda\sin\theta\sin^k\alpha]|dz| $$ again, and we see that everything boils down to the inequality $$ \cos(\theta-\alpha+\tfrac{4\alpha}{\rho})\le\cos(\theta-\alpha)\,. $$

If $\theta-\alpha>0$, then it is $\le\frac{\pi} 2-\alpha$ and the rotation is by at most $4\alpha\le 2\alpha+\pi$, so we get short of $\frac{3\pi}2+\alpha$, where we could have any chance to bring the cosine up again. Thus, we need to consider only the case $\theta-\alpha<0$. If $\rho\le 2$, then our rotation is by at least $2\alpha$ and at most $2\pi$, which diminishes the cosine of any angle between $-\alpha$ and $0$. Finally, if $\rho>2$, then we shall just show that we necessarily have $\theta>\alpha$ (the case we have considered already). Indeed, comparing the tangents, we see that we need just to prove the inequality $$ \left(\frac{\pi\rho}2-\alpha\right)\frac 1{\cos^2\alpha}\ge 2\tan\alpha\,. $$ Since $\rho>2$, that reduces to $$ \pi-\alpha>\frac\pi 2>1\ge \sin(2\alpha) $$ and the story is over.

The admissible contours are not unique, but the leeway in choosing them is rather small. I just posted the most elegant ones I could find. I hope it is all correct, but feel free to ask questions if something looks fishy or is just unclear.