Improving Newton's iteration where the derivative is near zero?
In most cases this works fine, however if the y is in the near of a maximum (or minimum) of f, where the shape is very similar to the maximum of a sinus-curve, the newton-iteration does not converge.
It doesn't converge in what way? Knowing the failure mode is important in fixing them. There are three main possibilities: the iteration is converging slowly/asymptotically, the iteration is converging to the wrong value (ie diverging from the answer you want), or the iteration is approaching a limit cycle (bouncing around among 2, 3, or more values repeatedly).
If your problem is that convergence near a minimum is too slow, and the second derivative doesn't vanish there, too, you could try Halley's method. The trouble with Halley's method is that when your state is far from the root it can cause you to overshoot, reducing your basin of convergence. So, I take a cue from "Numerical Recipes in C," when using it I clamp the effect so that it isn't too small or too large: \begin{align} x_{n+1} = x_n - \left(\frac{f_n}{f_n'}\right) \cdot \mathrm{CLAMP}\left(1 - \frac{f_nf_n''}{2[f'_n]^2}, \frac{1}{2}, 2\right)^{-1}. \end{align}
The ability to speed up convergence by using higher order methods in this class (Househoder's methods) is limited, however, by the number of derivatives that are zero. A simple example of a function that no Newton like method will find the root of quickly: $e^{-1/x^2}$, (all of its derivatives vanish at $x=0$).
If the problem is that the iteration is diverging from the root you want to find then there's a really easy answer: iterate in the other direction. See, in any iterative process like this you will generically have attractors (points the iteration will converged to, if the starting point is in its basin), repulsors (points the iteration will diverge from), and limit cycles (attractors of the double step, triple step, etc). I just recently had a problem with Newton's method always trying to converge to the solution I didn't want in a two solution problem. Flipping the sign to change the direction of iteration fixed that up nicely, because reversed iteration flips the roles of attractors and repulsors.
If the problem is that you're hitting a limit cycle, then that's more tricky. See, if $x_{n+1} = g(x_n)$ is your update rule, then you can define a fixed point as $g(x) = x$. Where the trouble begins is $g$ implies a whole family of update rules, call them $g^{[n]}$, that are the same as applying $g$ $n$ times. The fixed points of $g$ will all be fixed points of $g^{[n]}$, but $g^{[n]}$ will generically have more. Because iterating in $g$ is the same as iterating in $g^{[n]}$, the fixed points of $g^{[n]}$ can become the attractors that dominate the behavior of the sequence, pulling it into a limit cycle of length $n$. If we assume, as is often the case, that the fixed point you want is a reuplsor within the basin of attraction of a limit cycle, then you might be able to find that point if you: 1, detect the cycle; 2, synthesize a guess that is not to close to the limit cycle points (e.g. average them, or use the $x_0$ that converged to the cycle in the first place); and 3, iterate backwards.
I can't offer any guarantees without further information, though, because those basins of attraction are generically fractals in shape.
If you have some starting point $x_0$, you could obtain $x_1$ by $$x_n=x_{n-1}-\frac{f'(x_{n-1})}{f''(x_{n-1})}$$ and after that use $$x_n=x_{n-1}-\alpha\frac{f'(x_{n-1})}{f''(x_{n-1})}$$ where $\alpha$ solves $$x_{n-1}-\alpha\frac{f'(x_{n-1})}{f''(x_{n-1})}=x_{n-2}-\alpha\frac{f'(x_{n-2})}{f''(x_{n-2})}$$ For the following function with many $0$-valued derivatives at the minimum $x=1$,
$f(x)=4 (1-x)^8+4 (1-x)^{12}+1888 (1-x)^{14}+1888 (1-x)^{22}$
when starting at $x_0 = 0$ the 3rd iteration overshoots big time, but sufficiently close to the minimum the quadratic convergence begins (see bottom picture, it actually looks super-quadratic) despite 6 derivatives being equal to $0$.
In the case where all derivatives are 0 at the minimum the modification doesn't make the convergence quadratic. I haven't found something that really works in that case.
In higher dimension, where $x_n$ and the fraction are vectors, you can solve the same alpha equation for each parameter, or just those that cause slow convergence.
