Find a such that $ax^{17}+bx^{16}+1$ is divisible by $x^2-x-1$.

Hint. If $x=\frac{1±\sqrt{5}}{2}$ then $x^2=x+1$ and \begin{align} ax^{17}+bx^{16}+1 &=(ax+b)(x+1)^8+1 \\ &=(ax+b)(x^2+2x+1)^4+1\\ &=(ax+b)(3x+2)^4+1\\ &=(ax+b)(21x+13)^2+1. \end{align} Can you take it from here? Note that 3,2, 21,13 are all Fibonacci numbers.

The given divisor has roots $\varphi=\frac{1+\sqrt5}2$ and $-\frac1\varphi$, so the $17$th-degree polynomial must also have these roots. This gives a system of equations for $a$ and $b$, which can then be solved.

The high exponents can be simplified by using the property $\varphi^2=\varphi+1$: $$\varphi^4=3\varphi+2$$ $$\varphi^8=21\varphi+13$$ $$\varphi^{16}=987\varphi+610$$ $$\varphi^{17}=1597\varphi+987$$ Thus $$a\varphi^{17}+b\varphi^{16}+1=0\implies a(1597\varphi+987)+b(987\varphi+610)=-1$$ $$-a\varphi^{-17}+b\varphi^{-16}+1=0\implies -a+b\varphi=-1597\varphi-987$$ We can see that $a=987,b=-1597$ is a solution to this system, and hence the original problem.

As you observed correctly, the 17th degree polynomial must also have the same roots ($\frac{1\pm\sqrt5}2$). Substituting these roots into $ax^{17}+bx^{16}+1=0$, you get two equations which can be solved simultaneously for $a$ and $b$, yielding $$a=987 \qquad \text{and} \qquad b=-1597.$$