Stable Nuclei - Deviation from equal protons and neutrons
Well, look at the other terms...in particular consider the Coulomb term as that one depends only on the proton number.
Walecka's book writes it as $$ E_3 = \frac{3}{5} \frac{Z (Z-1)}{4\pi R_C} e^2 \approx a_3 \frac{Z^2}{A^{1/3}} \,.$$ This term is strictly positive and grows rapidly as the atomic number increases, while being slightly decreased by growing neutron number.
At some point this contribution begins to outweigh the energy demands of the neutron degeneracy.