Anti-gravity in an infinite lattice of point masses
This is not correct, but Newton believed this. The infinite system limit of a finite mass density leads to an ill defined problem in Newtonian gravity because $1/r^2$ falloff is balanced by density contributions of size $r^2\rho$, and there is no well defined infinite constant-mass-density system. The reason is that there is no equilibrium of infinite masses in Newtonian gravity--- you need an expanding/contracting Newtonian big-bang.
This is subtle, because symmetry leads you to believe that it is possible. This is not so, because any way you take the limit, the result does not stay put. This was only understood in Newtonian Gravity after the much more intricate General Relativistic cosmology was worked out.
I think that your initial intutiion is right--before the point particle is removed, you had (an infinite set of) two $\frac{G\,m\,m}{r^{2}}$ forces balancing each other, and then you remove one of them in one element of the set. So initially, every point particle will feel a force of $\frac{G\,m\,m}{r^{2}}$ away from the hole, where $r$ is the distance to the hole. An instant after that, however, all of the particles will move, and in fact, will move in such a way that the particles closest to the hole will be closer together than the particles farther from the hole. The consequence is that the particles would start to clump in a complicated way (that I would expect to depend on the initial spacing, since that determines how much initial potential energy density there is in the system)
I assume by square lattice you mean a 3D cubic lattice because there's no translational symmetry along the $z$-axis for a 2D square lattice.
Suppose the masses are located at $(n_x, n_y, n_z)$ where $n_{x,y,z}\in\mathbb Z$. Let's also define the unit of mass and length so that $m=l=1$.
Consider the total force acted on the mass point at (0, 0, 0) just due to the 1st octant $(x>0,y>0,z>0)$: $$\begin{aligned} \mathbf F_{+++} &= -G \sum_{n_x=1}^\infty \sum_{n_y=1}^\infty \sum_{n_z=1}^\infty \frac{n_x \hat{\mathbf x} + n_y \hat{\mathbf y} + n_z \hat{\mathbf z}}{(n_x^2+n_y^2+n_z^2)^{3/2}} \\ &= -G \left( \hat{\mathbf x} \sum_{n_x=1}^\infty \sum_{n_y=1}^\infty \sum_{n_z=1}^\infty \frac{n_x}{(n_x^2+n_y^2+n_z^2)^{3/2}} + \dotsb \right), \end{aligned}$$ however, the sum actually diverges, since, $$\begin{aligned}\sum_{n_x=1}^\infty \sum_{n_y=1}^\infty \sum_{n_z=1}^\infty \frac{n_x}{(n_x^2+n_y^2+n_z^2)^{3/2}} &\ge \int_1^\infty \int_1^\infty \int_1^\infty \frac{n_x}{(n_x^2+n_y^2+n_z^2)^{3/2}} dn_x dn_y dn_z \\ &= \int_1^\infty \int_1^\infty \frac1{\sqrt{1+n_y^2+n_z^2}} dn_y dn_z \\ &= \infty, \end{aligned} $$ so while symmetry suggests that the force at center is 0, mathematically it is not well defined.
Of course, if we assume the net force can be well-defined as 0 (e.g. the gravity actually decays faster than $1/r^3$!), then Points 1 and 3 are correct. When we remove a particle from the lattice, the contribution $-\frac{GMm\hat{\mathbf r}}{r^\alpha}$ will be subtracted from it, so it is as if there is a particle of negative mass $-m$ put to that point. This is because force is additive and gravity is proportional to mass.
(Yeah this is stating the obvious.)