Prime Numbers And Perfect Squares

If $p^2+q^2+7pq = r^2$ ($r$ being any integer), then $(p+q)^2 + 5pq = r^2$. So $5pq = r^2-(p+q)^2 = (r+p+q)(r-p-q)$.

Since $p, q$ and $5$ are all prime, it follows that one of the factors on the right-hand side is equal to one of them, and the other factor is the product of the other two. As clearly visible, $r+p+q$ is greater than any of the numbers $p, q$ and $5$. So it must be the product of two of those numbers (maybe three too), and the other factor $r-p-q$ must be equal to $p, q$ or $5$ (or $1$).

Now, different cases arise:

CASE $1$:

If $r-p-q = p$ then $r=2p+q$, and thus original equation becomes $p^2+q^2+7pq = (2p+q)^2$, which simplifies to $p=q$, Same if $r-p-q = q$.

CASE $2$: If $r-p-q = 5$ then $r = p+q+5$, and the equation $5pq = (r+p+q)(r-p-q)$ becomes $pq = 2(p+q)+5$. You can write this as $(p-2)(q-2)=9$, and the only solutions to this are $p=q=5$ and $p=3;q=11$ (or the other way round).

CASE $3$: $r+p+q$ might be equal to the product of all three numbers $5pq$, with $r-p-q = 1$. But then the equation becomes $2p+2q+1 = 5pq$, which is clearly impossible because the right-hand side is visibly greater than the left-hand side (Though there are solutions like $(1,1)$, but $1$ is not prime ).

So, to summarise the whole answer, We can say that only solutions are:

$(p,q)=(p,p),(3,11),(11,3)$

SOURCE

Assume $p^2+7pq+q^2=n^2$ with $p\ge q$. Note that $p^2+2pq+p^2=(p+q)^2$, hence $$(n+p+q)(n-p-q)=n^2-(p+q)^2=5pq$$ We know the prime factorization of $5pq$ and that $n-p-q<n+p+q$, hence conclude that $n-p-q$ is $\in\{1,5,q,p\text{ (if $5q>p$)},5q\text{ (if $5q<p$)}\}$. Inverstigate these cases one by one.

Since the equation $p^2+7pq+q^2=n^2$ is symmetric in $p$ and $q$, we may assume without loss of generality that $p\lt q$ (setting aside the obvious solutions with $p=q$). For reasons that will become clear in a moment, it's easy enough to check for solutions with $p\lt q\le7$ (there are none).

Multiplying both sides of the equation by $4$, it's possible to rewrite it as

$$(2p+7q+2n)(2p+7q-2n)=45q^2$$

Now the prime $q$ cannot divide both factors on the left hand side, since the sum of those two factors is congruent to $4p$ mod $q$. But $2p+7q-2n\lt9q\lt q^2$, so we must have $2p+7q+2n=aq^2$ and $2p+7q-2n=b$ with $ab=45$. Multiplying each of these by $a$ and summing (to eliminate $n$), we have

$$4ap+14aq=a^2q^2+45$$

or

$$4ap+4=a^2q^2-14aq+49=(aq-7)^2$$

which solves to

$$q={7+2\sqrt{ap+1}\over a}$$

(We can disregard the negative square root, since $q\gt7$.) This implies $ap+1$ is a square, which clearly implies $ap+1=k^2$ for some integer $k\ge2$ (since $ap\ge2$). But now the inequality $q\gt p$ becomes $7+2k\gt k^2-1$, or $(k-1)^2\lt9$. Thus $k\lt4$, so there are just two possibilities: $k=2$ or $k=3$, correpsponding to $ap+1=4$ and $ap+1=9$. But since $p$ is primes and $a$ is a divisor of $45$ (hence odd), the only possibility is $p=3$ and $a=1$, which gives $q=7+2\sqrt4=11$. And indeed

$$3^2+7\cdot3\cdot11+11^2=361=19^2$$

And that's it: The only prime pairs $(p,q)$ for which $p^2+7pq+q^2$ is a square are $(3,11)$, $(11,3)$, and pairs of the form $(p,p)$.

Remark (added later, after a restless night's sleep): When I first wrote up this answer, I did the scratchwork to check that there are no solutions with $p\lt q\le7$. This did the job, to be sure, but it felt a little unsatisfying. It only later (in the middle of the night) occurred to me to think mod $8$: If $p\lt q$, then $q$, and hence $n$, are both odd, so that $q^2\equiv n^2\equiv1$ mod $8$, which reduces the equation to $p(p-q)\equiv0$ mod $8$. From this it's easy to see that $p\equiv q$ mod $8$, which implies $q\ge2+8=10$.