Rate of convergence of the prime zeta function P(2)

Indeed, this is a type of question that is standard to those in the know (pretty much exactly what MathOverflow is for!). The famous paper of Rosser and Schoenfeld contains many estimates for $\pi(x)$, the number of primes up to $x$: for example, their Corollary 1 tells us that $$ \frac x{\log x} < \pi(x) < 1.25506 \frac x{\log x} \text{ for } x\ge17. $$ Bounds for your sum (for $x\ge17$ in this case) can then be found using partial summation. For example, for the lower bound, \begin{equation*} \begin{split} \sum_{p>x} \frac1{p^2} &= \int_x^\infty \frac1{t^2} \,d\pi(t) \\ &= \frac{\pi(t)}{t^2}\bigg|_x^\infty + \int_x^\infty \pi(t) \frac2{t^3} \,dt \\ &> \frac1{t\log t}\bigg|_x^\infty + \int_x^\infty \frac2{t^2\log t} \, dt \\ &= -\frac1{x\log x} - 2\mathop{\rm li}\bigg( \frac1x \bigg), \end{split} \end{equation*} where $\mathop{\rm li}(t) = \int_0^t dt/\log t$ is the logarithmic integral function. [The second expression on the top line is a Riemann–Stieltjes integral; if you're not familiar with those, one can verify by hand that the sum equals the expression on the second line.) This last expression can be shown to be asymptotic to $1/(x\log x)$ for large $x$.

The calculation of the upper bound is exactly the same except with an extra factor of $1.25506$. Indeed, a more careful calculation (using sharper bounds from Rosser and Schoenfeld from earlier on the same page) will give both upper and lower bounds that are asymptotic to $1/(x\log x)$.