The coefficient and asymptotic in generating function

Part (b) $$[x^n] \frac{1}{\sqrt{1-6x + x^2}} = [x^n] \frac{1}{\sqrt{1-2xt + x^2}}\big|_{t=3}= [x^n]\sum_{n=0}^\infty P_n(3) x^n $$ where the generating function of the Legendre polynomials has been used. The exact answer is thus $P_n(3).$ Look up the asymptotics for the Legendre polynomials and you'll find

$$ P_n(t) \sim \frac{1}{\sqrt{2 \pi n}} \frac{ (t+\sqrt{t^2-1})^{n+1/2} }{(t^2-1)^{1/4}}$$ when $t$ is large (outside the oscillatory region.) Putting in $t=3$ and comparing the asymptotic formula to the exact, we find that, for $n=50,$ the asymptotic formula exceeds the actual by only 0.23%.

The instinct for paying attention to where the singularities are, at $3\pm\sqrt{8},$ (there's a typo in some of your problem setup) is a good one. The penultimate equation utilizes the largest root. Making this approach rigorous is often called 'singularity analysis.' I've had Legendre polynomials on the brain for a while, so I didn't want to reinvent a well-known result (I think the above equation can be found in Bender and Orszag.) Ultimately, an integral representation for the Legendre polynomials is analyzed using the well-known saddle-point technique.

We show the following asymptotic expansion of $A(x)=\frac{1}{\sqrt{1-6x+x^2}}$ is valid: \begin{align*} \color{blue}{[x^n]A(x)\sim\frac{\left(3+2\sqrt{n}\right)^n}{2\sqrt{\left(3\sqrt{2}-4\right)\pi n}} \left(1-\frac{8-3\sqrt{2}}{32n}+\mathcal{O}\left(\frac{1}{n^2}\right)\right)}\tag{1} \end{align*} $A(x)$ has the isolated singularities \begin{align*}x_0=3-2\sqrt{2}\qquad x_1=3+2\sqrt{2}, \qquad x_0x_1=1 \end{align*} with $x_0$ being the dominant singularity determining the asymptotic expansion (1).

Expansion at $x_0=3-2\sqrt{2}$:

At first we consider \begin{align*} A(x)&=\frac{1}{\sqrt{1-6x+x^2}}=\frac{1}{\sqrt{\left(x-x_0\right)\left(x-x_1\right)}}\\ &=\frac{1}{\sqrt{x_0x_1}}\,\frac{1}{\sqrt{\left(1-\frac{x}{x_0}\right)}}\,\,\frac{1}{\sqrt{\left(1-\frac{x}{x_1}\right)}}\\ &=\frac{1}{\sqrt{1-\left(3-2\sqrt{2}\right)x}}\,\frac{1}{\sqrt{1-\left(3+2\sqrt{2}\right)x}}\tag{2} \end{align*}

We expand (with some help of Wolfram Alpha) the first term in (2) at $x_0$ and obtain \begin{align*} \frac{1}{\sqrt{1-\left(3-2\sqrt{2}\right)x}}&=\frac{1}{2\sqrt{3\sqrt{2}-4}}+\frac{1}{16\sqrt{2}\sqrt{3\sqrt{2}-4}}\left(x-\left(3-2\sqrt{2}\right)\right)\\ &\qquad+\mathcal{O}\left(\left(x-\left(3-2\sqrt{2}\right)\right)^2\right)\\ &=\frac{1}{2\sqrt{3\sqrt{2}-4}}-\frac{3-2\sqrt{2}}{16\sqrt{2}\sqrt{3\sqrt{2}-4}}\left(1-\left(3+2\sqrt{2}\right)x\right)\\ &\qquad+\mathcal{O}\left(\left(1-\left(3+2\sqrt{2}\right)x\right)^2\right)\tag{3} \end{align*}

The other term $\frac{1}{\sqrt{1-\left(3+2\sqrt{2}\right)x}}$ in (2) is already in the expanded form we need.

Asymptotic expansion:

We follow chapter VI in Analytic Combinatorics by P. Flajolet and R. Sedgewick and get from Theorem VI.1 an asymptotic expansion of \begin{align*} [x^n]f(x)=(1-x)^{-\alpha}\qquad\qquad\alpha\in\mathbb{C}\setminus\mathbb{Z}_{\leq 0}\tag{4} \end{align*} We take terms up to order $2$ and get \begin{align*} [x^n]f(x)\sim\frac{n^{\alpha-1}}{\Gamma(\alpha)}\left(1+\frac{\alpha(\alpha-1)}{2n}+\mathcal{O}\left(\frac{1}{n^2}\right)\right)\tag{5} \end{align*} We need from (5) the following expansions \begin{align*} [x^n](1-x)^{-\frac{1}{2}}&\sim\frac{n^{-\frac{1}{2}}}{\Gamma\left(\frac{1}{2}\right)} \left(1+\frac{\left(\frac{1}{2}\right)\left(-\frac{1}{2}\right)}{2n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\\ &=\frac{1}{\sqrt{n\pi}}\left(1-\frac{1}{8n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\tag{6}\\ [x^n](1-x)^{\frac{1}{2}}&\sim\frac{n^{-\frac{3}{2}}}{\Gamma\left(-\frac{1}{2}\right)} \left(1+\frac{\left(-\frac{1}{2}\right)\left(-\frac{3}{2}\right)}{2n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\\ &=-\frac{1}{2\sqrt{n^3\pi}}\left(1+\frac{3}{8n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\\ &=-\frac{1}{2\sqrt{n\pi}}\left(\frac{1}{n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\tag{7} \end{align*}

Note, that (4) is given in standard form, but we have something like $(1-x_1x)^{-\alpha}$ instead. We overcome this by recalling a Taylor series expansion and noting that \begin{align*} [x^n]f(x)&=x_{1}^{-n}[x^n]f\left(x_1x\right)\\ [x^n]f(x_1x)&=x_1^n[x^n]f(x)\\ &=\left(3+2\sqrt{2}\right)^n[x^n]f(x)\tag{8} \end{align*}

Putting all together:

We take from (3) the expansion at $x_0$ and get \begin{align*} A(x)&=\frac{1}{\sqrt{1-\left(3-2\sqrt{2}\right)x}}\,\frac{1}{\sqrt{1-\left(3+2\sqrt{2}\right)x}}\\ &=\Bigg\{\frac{1}{2\sqrt{3\sqrt{2}-4}}-\frac{3-2\sqrt{2}}{16\sqrt{2}\sqrt{3\sqrt{2}-4}}\left(1-\left(3+2\sqrt{2}\right)x\right)\\ &\qquad\qquad+\mathcal{O}\left(\left(1-\left(3+2\sqrt{2}\right)x\right)^2\right)\Bigg\}\,\left(1-\left(3+2\sqrt{2}\right)x\right)^{-\frac{1}{2}}\\ &=\frac{1}{2\sqrt{3\sqrt{2}-4}}\left(1-\left(3+2\sqrt{2}\right)x\right)^{-\frac{1}{2}}\\ &\qquad-\frac{3-2\sqrt{2}}{16\sqrt{2}\sqrt{3\sqrt{2}-4}}\left(1-\left(3+2\sqrt{2}\right)x\right)^{\frac{1}{2}} +\mathcal{O}\left(\left(1-\left(3+2\sqrt{2}\right)x\right)^\frac{3}{2}\right) \end{align*} Using (6) and (7) together with (8) we obtain \begin{align*} \color{blue}{[x^n]A(x)}&\color{blue}{\sim}\frac{\left(3+2\sqrt{2}\right)^n}{2\sqrt{3\sqrt{2}-4}} \frac{1}{\sqrt{n\pi}}\left(1-\frac{1}{8n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\\ &\qquad+\frac{\left(3+2\sqrt{2}\right)^n\left(3-2\sqrt{2}\right)}{32\sqrt{2}\sqrt{3\sqrt{2}-4}}\frac{1}{\sqrt{n\pi}}\left(\frac{1}{n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\\ &=\frac{\left(3+2\sqrt{2}\right)^n}{2\sqrt{\left(3\sqrt{2}-4\right)\pi n}}\left(1-\frac{1}{8n} +\frac{3-2\sqrt{2}}{16\sqrt{2}n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)\\ &\,\,=\color{blue}{\frac{\left(3+2\sqrt{2}\right)^n}{2\sqrt{\left(3\sqrt{2}-4\right)\pi n}}\left(1 -\frac{8-3\sqrt{2}}{32n}+\mathcal{O}\left(\frac{1}{n^2 }\right)\right)} \end{align*} and the claim (1) follows.

Note: The expansion (1) can be found e.g. in Asymptotics of the weighted Delannoy numbers by R. Noble.