If we remove all electrons from a conductor, how can the positive charge rearrange itself?

If you suddenly removed all the electrons from a piece of material, or even just the valence electrons, you would be left with a huge concentration of positive ions in a small volume, which would exert a huge electrostatic repulsion on each other. Since you no longer have the bonding influence of the electrons to counteract this repulsion, the material would blast apart, in very short order, in a process that's known as a Coulomb explosion.

To put some numbers into things, suppose that you have one cubic millimeter of iron, and you suddenly remove one electron per atom. This turns out to be about $0.00014\:\mathrm{mol}$ of iron, but because Avogadro's number is so huge, that's about $8.491\times 10^{19}$ electrons, and a corresponding charge of about $13.6\:\rm C$ in the sphere, an electrostatic charge distribution that holds about $1.6\times 10^{15}\:\rm J$ of energy, or about $385$ kilotons of TNT, i.e. about twenty times bigger than the explosion that flattened Hiroshima.

(And, obviously, that's the amount of energy that you will need to put in to be able to suddenly remove all of those electrons. In more human terms, that's a $1\:\rm GW$ power station running nonstop for 18 days. And, as mentioned in the comments, this amount of energy represents about twenty times more than the original rest mass of the iron.)

That said, if you scale things down significantly, then Coulomb explosions can become quite reasonable things and indeed important research tools. Normally you do this with small(ish) molecules and atomic clusters (so, from a few to a few hundred atoms), where you have a few hundred electrons or so (instead of tens of quintillions), and you remove them with a high-intensity, high-photon-energy beam coming from a free-electron laser (FEL). In the process you might then get single-molecule x-ray diffraction spectra, information about the initial structure from where the atoms flew off to after the explosion, or you might just learn about the physics of the ionization and explosion processes. For a nice overview, see these slides by Christoph Bostedt, or the papers in this google search.

You will not be able to remove all the electrons of any conductor no matter what kind... The removal of electrons from any conductor comes from the valence electrons of the atoms in particular from the one electron most weakly connected to the nucleus.

As you keep going, removing more and more electrons from the conductor, I think you should reach a point on which, the conductor will break apart as far as the interatomic/molecular forces that keep their atoms/molecules together are overwhelmed by the coulombian force between the positive charges in this atoms/molecules.

The key to the answer is where you equate a lack of electrons with an abundance of protons. That is a very misleading analogy. The correct analogy is actually to equate electrons (carriers of a negative charge) with holes (which is the absence of an electron where one should be. Holes are positively charged).

Protons are fixed in place (at least in a solid, and if you ignore Brownian motion and the like). They are "frozen" into the nucleus of each atom.

One gram of copper, or one centimeter of copper wire, contains a specific number of atoms, and therefore a specific number of protons, and that doesn't change no matter what.

Now in a neutral substance, there is one electron for each proton on average. Individual atoms can lose one electron, or even two, and that makes them positive ions. However, when that happens, these ions have a strong attractive force on electrons. Removing the first electron from a neutral atom tends to be easy (in a conductor). Removing a second one becomes more difficult, and removing a third electron, or even more, becomes incrementally more difficult, and eventually impossible simply because the atom becomes more and more positively charged, and therefore attractive to electrons.

But hypothetically assume that you could remove all electrons from your conductor, and you could prevent electrons from the surrounding air to get back in. The first thing that would happen is that all the remaining atomic nuclei would repel each other. Your conductor would disintegrate (probably with a huge explosion).

Now if you could prevent that, too, you still couldn't have an electric field. An electric field exists between two charges. An electron does not "have" an electric field. A proton does not have an electric field.

So if you could remove all electrons from a conductor (which you can't), and could prevent it from flying apart (which you can't), there would be nothing left that could generate an electric field.