Partial fractions and using values not in domain

When you get to the stage $3x-8 = A(x+1) + B(x-5)$, you've forgotten about the partial fractions and all you're trying to do is determine a polynomial identity. i.e: what $A$ and $B$ will make $A(x+1) + B(x-5)$ identical to $3x-8$ for all values of $x$, this includes $x=-1$ and $5$.

Once you've got those values, then you divide both sides of the polynomial identity by $(x+1)(x-5)$ with the added restriction that $x\neq -1, 5$. So this gives you $$\frac{3x-8}{(x+1)(x-5)} = \frac{A(x-5) + B(x+1)}{(x+1)(x-5)} = \frac{A}{x-5} + \frac{B}{x+1}$$

which holds for all $x \neq -1, 5$. (which is what let us do the cancelling bit on the RHS)

---

Some explanation: the motivation for using $x=-1$ and $5$ is that is efficient in the sense that it makes one of the terms in the identity $0$, speeding up the process of finding the values of the coefficients. If you wanted to, you could use any other two values such as $x=3$ and $4$ to get two simultaneous equations in $A$ and $B$, allowing you to determine their values, just not as efficiently.