Why is the potential energy of a spring the same when it is compressed and stretched?

One way is to explain how a spring actually works.

A coil spring is a large wire that is wound into a helix. When you compress or extend a spring, from the wire’s perspective, you aren’t really pushing or bending. Instead, you are twisting the wire one way or another.

Twisting a bar clockwise or counterclockwise should be the same thing.

Unfortunately, there isn't a physical reason that the energy must be the same at +d and -d, because in general it is not. All springs will be nonlinear and non-symmetric if you stretch them enough. The reason we can write $F=-kx$ is that you can always linearize the force for sufficiently small displacements, and then you just assume you're working in the linear regime. You might just have to assert to the students that for small enough displacements, the energies are the same in each direction. Alternatively, you could use another system for which there would be no reason to believe that the energies aren't symmetric, such as a pendulum.

One aspect you might mention is that the potential energy of a particle is related to the force the particle feels (since mathematically the force is the negative gradient of the potential energy). This point is powerful for intuitive arguments. For example, you can argue that the energy shouldn't be |d| because then the restoring force would be the same whether the spring is stretched a little or a lot, which seems unreasonable.

If possible, you could also have students test this with a spring in the classroom, feeling the force by holding the spring displaced by different amounts in either direction to get them on board with the idea that the force is symmetric on either side of the equilibrium point and that it increases as you pull (or push farther).

This is really a question of practical vs. theoretical. Any spring actually will have a non-linear effect. But in theory we ignore it for a basic understanding. So we assume it is linear. Here are some ways to approach this: If the spring body is hidden from your eyes and all you have access to is a movable handle connected to the free end of the (hidden) spring, then you can't tell which way the spring distorts (stretch or compress) when you move that handle--it takes the same amount of energy to distort it the same distance. Besides that, consider that a short section of the spring (for example, a short section of the spring wire, assuming a coil spring), bends one way when the spring is stretched and the other way when the spring is compressed. Bending a piece of metal compresses one side and stretches the other, or stretches the one side and compresses the other. If the spring is homogeneous, then this is the same sort of distortion.