Are there infinitely many palindromes whose sum is a perfect square?

Yes, there are infinitely many. For any $n\in\mathbb{N}$ let $$\begin{align} a_n&=3+4\times10^{n}+7\times10^{2n}+4\times10^{3n}+3\times10^{4n}\\ b_n&=4+2\times10^{n}+8\times10^{2n}+2\times10^{3n}+4\times10^{4n}\\ c_n&=5+4\times10^{n}+11\times10^{2n}+4\times10^{3n}+5\times10^{4n} \end{align}$$ Then $a_n$ and $b_n$ are palindromes and $$ a_n^2+b_n^2=c_n^2. $$ Moreover, $\gcd(a_n,b_n)=1$. Let $$\begin{align} A_n&=7+8\times10^{n}+8\times10^{2n}+8\times10^{3n}\\ B_n&=-\frac12\bigl(10+21\times10^{n}+22\times10^{2n}+12\times10^{3n}\bigr) \end{align}$$ Then $$ A_n\,a_n+B_n\,b_n=1. $$

I found this identity doing a brute force search. Any palindrome with an even number of digits is divisible by $11$, so one (or both) of the $p_i$ must have an odd number of digits. I have searched for $p_1$ with $2\,k+1$ digits, $1\le j\le 7$. These are the results. Included are also some examples of $p_1$ with $17$ digits. $$ \begin{array}{ll} 313 & 48984 \\ 464 & 777 \\ 25652 & 55755 \\ 34743 & 42824 \\ 52625 & 80808 \\ 80308 & 5578755 \\ 2152512 & 575575 \\ 2532352 & 5853585 \\ 5679765 & 23711732 \\ 304070403 & 402080204 \\ 341484143 & 420282024 \\ 345696543 & 422282224 \\ 355949553 & 690019910096 \\ 359575953 & 401141104 \\ 27280108272 & 55873637855 \\ 3004007004003 & 4002008002004 \\ 3044529254403 & 4022208022204 \\ 3410048400143 & 4200028200024 \\ 3414249424143 & 4202028202024 \\ 3450569650543 & 4224448444224 \\ 6381414141836 & 778233332877 \\ 395734505437593 & 426982282289624 \\ 404990565099404 & 747709181907747 \\ 461781161187164 & 778676101676877 \\ 30004000700040003 & 40002000800020004 \\ 30040410801404003 & 40020200800202004 \\ 30044412921444003 & 40022200800222004 \\ 30081842624818003 & 40041401210414004 \\ 30401040804010403 & 40200020802000204 \\ 30405060906050403 & 40202020802020204 \\ 32682698889628623 & 46020004840002064 \\ 34100004840000143 & 42000002820000024 \\ 34104204940240143 & 42002002820020024 \\ 34140434943404143 & 42024404840442024 \\ 34505056965050543 & 42244646864644224 \\ \end{array} $$