Taylor polynomial: the higher the degree, the better the approximation?

I will assume without loss of generality that $x_0 = 0$ and we denote by $I$ an open neighborhood of $0$.

In general it is false that the approximation of the $n^{\text{th}}$ Taylor polynomial of a smooth function is getting better as $n$ grows. In proposition 1, we see that there exists a function such that on every neighborhood of $0$ the supremum of the Taylor remainder goes to infinity as $n \to +\infty$. However, for a smooth function you can always find a neighborhood of $0$ on which the $N$ first supremun of the Taylor remainder decrease but the width of your neighborhood depend on $N$, see proposition 2.

Proposition 1. There exists a function $f \in \mathcal{C}^\infty(I)$ such that for all $\varepsilon > 0$ and $[-\varepsilon,\varepsilon] \subset I$, we have $$ \lim_{n \to +\infty} \sup_{x\in [-\varepsilon,\varepsilon]} |T_n(x) - f(x)| = +\infty $$ where $T_n(x) = \sum_{k=0}^n \frac{f^{(k)}(0)}{k!}\, x^k$ is the $n^{\text{th}}$ order Taylor polynomial.

• First somewhat of a heuristic for the function $f$, for $x$ small enough, we have roughly $$ |T_n(x) - f(x)| \approx \frac{f^{(n+1)}(0)}{(n+1)!} |x|^{n+1} \quad\text{and}\quad |T_{n+1}(x) - f(x)| \approx \frac{f^{(n+2)}(0)}{(n+2)!} |x|^{n+2} $$ so the inequality $|T_{n+1}(x) - f(x)| \le |T_n(x) - f(x)|$ become $$ \frac{f^{(n+2)}(0)}{(n+2)!} |x|^{n+2} \le \frac{f^{(n+1)}(0)}{(n+1)!} |x|^{n+1} \quad\implies\quad |x| \le \frac{f^{(n+1)}(0)}{f^{(n+2)}(0)} (n+2). $$ If $\frac{f^{(n+1)}(0)}{f^{(n+2)}(0)} (n+2)$ goes to $0$ then you would not be able to find a neighborhood of $0$ to have your inequality for all $n$. To construct a counterexample, we will choose a function such that $$\lim_{n \to +\infty} \frac{f^{(n+1)}(0)}{f^{(n+2)}(0)} (n+2) = 0.$$.

• Proof of proposition 1. We choose a function $f \in \mathcal{C}^\infty(\mathbb{R})$ such that $f^{(n)}(0) = (n!)^2$, such a function exist by the Borel's lemma or by this post for an explicit construction for this particular example. We have that, for all $n \ge 0$, $$T_n(x) = \sum_{k = 0}^n k!\, x^k$$ and the series $\sum_{k\ge 0} k!\, x^k$ has a zero radius of convergence because, for $x \neq 0$, $\lim_{k \to +\infty} |k!\, x^k| = +\infty$. That mean that for $\varepsilon > 0$, there exists $a \in (0,\varepsilon)$ such that the sequence $(|T_n(a)|)_{n\ge 0}$ diverge to infinity. By the triangle inequality, we have $$ \sup_{x\in [-\varepsilon,\varepsilon]} |T_n(x) - f(x)| \ge |T_n(a) - f(a)| \ge |T_n(a)| - |f(a)| $$ then we obtain $$ \lim_{n \to +\infty} \sup_{x\in [-\varepsilon,\varepsilon]} |T_n(x) - f(x)| = +\infty. $$ This prove that your statement is false for general smooth function.

• Why do you graphically observe that it seem to work? The answers of @zhw. showed the following statement is true:

  Proposition 2. Let $f \in \mathcal{C}^\infty(I)$, for a fixed integer $N$ there exists $\varepsilon > 0$ such that $[-\varepsilon,\varepsilon] \subset I$ and for all $n \le N$, we have $$ \sup_{x \in [-\varepsilon,\varepsilon]} |T_{n+1}(x) - f(x)| \le \sup_{x \in [-\varepsilon,\varepsilon]} |T_n(x) - f(x)|. $$

WLOG $x_0=0.$ Claim: Given $n,$ there exists $r>0$ such that

$$|f(x)-T_{n+1}(x)| \le |f(x)-T_{n}(x)|,\,|x|<r.$$

To prove this, note first that if $f^{(n+1)}(0)=0,$ then $T_{n+1}=T_n,$ and there is nothing to prove. So assume $f^{(n+1)}(0)\ne 0.$ By the Langrange form of the remainder, we have

$$f(x)-T_{n}(x) = \frac{f^{(n+1)}(c)}{(n+1)!}x^{n+1},$$

where $c$ is between $0$ and $x.$

Because $f^{(n+1)}(0)\ne 0,$ there is $s>0$ and a positive constant $A$ such that $|f(x)-T_{n}(x)| \ge A|x|^{n+1}$ for $|x|<s.$ But using Lagrange again, we have the standard estimate

$$|f(x)-T_{n+1}(x)|=O(x^{n+2})$$

as $x\to 0.$ Since $O(x^{n+2})$ is bounded above by $A|x|^{n+1}$ for small $x,$ we have the desired result.

We can say more: either $f^{(n+1)}(0)= 0,$ in which case $T_{n+1}=T_n,$ or $f^{(n+1)}(0)\ne 0,$ in which case there will exist $r>0$ such that

$$|f(x)-T_{n+1}(x)| < |f(x)-T_{n}(x)|,\,0<|x|<r.$$

I've taken a few liberties in getting the main points across. Ask if you have questions.

I thought I would contribute another answer. This is on the problem of showing that even for an analytic function, there may be no $r>0$ such that $|f-T_{n}|\le|f-T_{n-1}|$ holds in $(-r,r)$ for all $n.$

Let $E=\{2,4,6,\dots\}.$ For $n\in E,$ define the polynmials

$$p_n(x) = \frac{x^n}{n^n}-\frac{x^{n+1}}{n^{n-1}}.$$

Now set $f(x)= \sum_{n\in E} p_n(x).$ This $f$ equals a power series convergent everywhere in $\mathbb R.$

Claim: For $n\in E,$ $|f(1/n)-T_{n-1}(1/n)| < |f(1/n)-T_{n}(1/n)|.$

Corollary: If $r>0,$ then $|f-T_{n}|\le|f-T_{n-1}|$ fails somewhere in $(-r,r)$ for all even $n>1/r.$

Lemma: For $n\in E,$

$$\tag 1\sum_{k=1}^{\infty}|p_{n+2k}(1/n)|<\frac{2}{n^{2n}}\frac{1}{n^4}\frac{1}{1-1/n^4}$$

Proof: For each $k$ we have

$$|p_{n+2k}(1/n)| \le \frac{(1/n)^{n+2k}}{(n+2k)^{n+2k}} + \frac{(1/n)^{n+2k+1}}{(n+2k)^{n+2k-1}} $$ $$< \frac{1}{n^{n+2k}}\frac{1}{n^{n+2k}} + \frac{1}{n^{n+2k+1}}\frac{1}{n^{n+2k-1}} = \frac{2}{n^{2n+4k}}.$$

Now sum over $k$ and recognize the geometric series to get the right side of $(1).$

Proof of claim: Note that

$$f(1/n)-T_{n-1}(1/n) = p_n(1/n) +p_{n+2}(1/n) + \cdots.$$

Now $p_n(1/n)=0.$ (That's why I chose the power series the way I did!) From the Lemma we then see

$$\tag 2 |f(1/n)-T_{n-1}(1/n)| < \frac{2}{n^{2n}}\frac{1}{n^4}\frac{1}{1-1/n^4} < \frac{2}{n^{2n}}\frac{1}{16}\frac{16}{15} = \frac{2}{15}\frac{1}{n^{2n}}.$$

On the other hand,

$$f(1/n)-T_{n}(1/n) = -\frac{(1/n)^{n+1}}{n^{n-1}} +p_{n+2}(1/n) + \cdots.$$

Thus, using the Lemma again,

$$ |f(1/n)-T_{n}(1/n)| \ge \left|\frac{(1/n)^{n+1}}{n^{n-1}}\right| -|p_{n+2}(1/n)| - |p_{n+4}(1/n)| - \cdots $$ $$\ge \frac{1}{n^{2n}} -\frac{2}{n^{2n}}\frac{1}{n^4}\frac{1}{1-1/n^4}$$ $$\tag 3\ge\frac{1}{n^{2n}} -\frac{2}{15}\frac{1}{n^{2n}}=\frac{13}{15}\frac{1}{n^{2n}}.$$

Thus $(2)<(3)$ and the claim is proved.