Find ALL integer solutions (n,m) of the equation $(n+1)(2n+1)=10m^2$

Let $x=n+1$. The equation becomes $$x(2x-1)=10m^2 \\ 2x^2-x-10m^2=0 \\ x=\frac{1 \pm \sqrt{1+80m^2}}{4}$$

It follows that $1+80m^2$ must be a perfect square. Thus $$y^2=1+80m^2 \Rightarrow\\ y^2-80m^2=1$$

This is the Pell equation, and has solutions. Now, for each solution $(y,m)$, $y$ must be odd. Then, each of $1+y, 1-y$ is even, and their sum is $2$, meaning that exactly one of them is divisible by $4$ (look at them $\pmod{4}$).

That particular value leads to an integer value of $x$.

I got this. We have $(n+1)(2n+1)=2n^2+3n+1=10m^2$, if we multiply last equation by $8$ we get $16n^2+24n+8=80m^2$, but we can write this equation as $$(4n+3)^2-1=80m^2.$$

So if we set $4n+3=x$ we get the Pell-equation $x^2-80m^2=1$. The fundamental solution of this equation is $x=9$ and $m=1$, hence we have the following recursive formulas $$x_{r+1}=9x_r+80m_r,$$ $$m_{r+1}=x_r+9m_r,$$ where $r\in\Bbb{N}$. Therefore $$n=\frac{9x_r+80m_r-3}{4}=\frac{9x_r-3}{4}+20m_r.$$

This means that in order to have $n\in\Bbb{Z}$ we must have $9x_r\equiv 3\pmod 4$, equivalently $x_r\equiv 3\pmod 4$. Now, from $x_{r+1}=9x_r+80m_r$ it's easy to prove by induction that $x_r\equiv 1\pmod4$ for every $r\in\Bbb{N}$, so $-x_r\equiv 3\pmod 4$ and this lead us to infinitely many solutions, i.e. for every $x_r$ which is a solution of $x^2-80m^2=1$, we just need to take $-x_r$ in order to have $n\in\Bbb{Z}$. Hence all the solutions are $$(n,m)=\Bigl(\frac{-x_r-3}{4}, \pm m_r\Bigr).$$