Gravitational waves in the Lorenz gauge?

The linearized theory is invariant up to the gauge changes $$\bar{h}_{\mu\nu} \to \bar{h}_{\mu\nu} + 2 \xi_{(\mu,\nu)} - \xi^\alpha_{\;,\alpha} \eta_{\mu\nu}.$$ By working in the Lorenz gauge $\bar{h}_{\mu\nu}^{\;\;\;,\nu} =0$ we still have a residual gauge freedom in the form of $\xi^\mathrm{R}_\nu$ which satisfy the condition $$(2 \xi^\mathrm{R}_{(\nu,\mu)}- \xi^{\mathrm{R}\alpha}_{\;\;\;,\alpha} \eta_{\mu\nu})^{,\nu}=0$$ which leads to the simple condition $$\Box \xi^\mathrm{R}_\nu = 0.$$ You can see that this also implies $$\Box (2 \xi^\mathrm{R}_{(\nu,\mu)}- \xi^{\mathrm{R}\alpha}_{\;\;\;,\alpha} \eta_{\mu\nu}) = 0$$ and your residual gauge freedom in the retarded potential formula is exactly $\phi_{\mu\nu} = 2 \xi^\mathrm{R}_{(\nu,\mu)}- \xi^{\mathrm{R}\alpha}_{\;\;\;,\alpha} \eta_{\mu\nu}$.

The reason why some people would write the retarded potential formula with $\phi_{\mu \nu} = 0$ is probably because they assume the reader understands that it is valid only up to the residual gauge freedom. At later points, this gauge is changed to shift to the TT gauge (i.e. $\phi_{\mu \nu}$ becomes nonzero).


As for the question of boundary conditions, we can indeed fix $\phi_{\mu\nu} = 0$ by assuming that the matter content is isolated and finite, and by requiring $\bar{h}_{\mu \nu} \to 0$ at infinity and $\Box \xi_\nu = 0$ everywhere. This is because $\xi_\nu$ can then be just superpositions of plane waves.

The funny thing, however, is that in the use of TT gauge to derive the quadrupole formula we also have $\bar{h}_{\mu \nu} \to 0$ at infinity but this gauge is different from the "Green-function gauge" $\phi_{\mu\nu} = 0$. We can do that because in that case we are using a gauge transformation which has $\Box \xi^\mathrm{R}_\nu \neq 0$ at the coordinate origin. I.e., $\bar{h}_{\mu \nu}^{\;\;\;,\nu} = 0$ in the TT gauge is valid only away from the origin and this is achieved by $\xi^\mathrm{R}_\mu$ being set equal to something like the electromagnetic quadrupolar radiation field.

The only case in which the TT gauge can be applied globally in the whole space-time along with the Lorentz condition is the case of a planar gravitational wave. In other cases such as the gravitational field of an isolated matter source, you will always have to leave out coordinate patches to apply that gauge choice.