What happens to the potential energy of a compressed spring when it dissolves in acid?

Your first assumption, that there would be a weakness in the material of the spring and it would suddenly break if corroded enough is what realistically would happen.

In your idealized situation each atom dissolved in the liquid had a proportional part of the potential energy of the stressed spring. As it looses its bonds with the surface, bonding with the molecules that corrode the material, its potential energy becomes kinetic energy of the new molecule and, due to the large number of atoms, it will be small enough to just turn into the heat energy of the liquid. In this process the potential energy of the spring will be gradually transformed to heat energy of the liquid, as is your guess.

Start by considering a gas phase reaction between two reagents $\mathrm{A}$ and $\mathrm{B}$. If you were watching the molecules with a microscope you'd see the reagents $\mathrm{A}$ and $\mathrm{B}$ collide, and the products $\mathrm{C}$ and $\mathrm{D}$ emerging. If you measure the kinetic energy of the product molecules you'll find it differs from the kinetic energy of the reagents by the energy of the reaction. So, if the reaction is exothermic the products will have greater kinetic energy than the reagents. When we talk about the energy of a reaction we mean the increased kinetic energy of the products.

Now, reactions in solution are a whole lot messier, but the basic idea remains the same. If you dissolve a spring not in tension you have a reaction something like:

$$\mathrm{M + 2H^+ \to M^{2+} + H_2}$$

(for a divalent metal dissolving in acid) and the reaction energy will be (initially) carried away by the $\mathrm{M^{2+}}$ ion and $\mathrm{H}$ atoms. I say initially because, of course, the reagents are not interacting in isolation because they're surrounded by solvent molecules, but basically the reaction energy is carried away as kinetic energy. In a liquid this, of course, raises the temperature.

Now, when you put the spring under compression, you are pushing metal atoms together. So when they react they carry away not just the energy of the reaction but also a small amount of energy from the fact the metal atoms will spring apart from each other when they can. The total energy of the reaction products will be slightly greater than if the spring was relaxed.

So, by compressing the spring you are very slightly increasing the reaction energy with whatever is dissolving the metal, and the result will be that the solution ends up very slightly hotter. Needless to say, in practice the increase would be immeasurably small.