How does this baby rattle work?

As you said, it's probably not magnetism if the balls are free to rotate; there is no reason they wouldn't just flip over and stick together, north to south. You can test this by buying some of those toy magnetic balls. The repulsive configurations are highly unstable and turn attractive with the slightest touch.

I'm going to go out on a limb and say it's static electricity; the balls are picking up charge by rubbing against the plastic. The electrons will always go to whichever material 'pulls' them harder (according to the triboelectric series), so the balls all get the same charge and hence repel.

I don't have a baby rattle with me but here are some ways you could test it:

• If you don't move the rattle for a while, the balls should come together as the static dissipates
• The effect should come back once you shake the rattle a couple times
• The effect should be smaller on humid days where static dissipates faster
• Another statically charged object should attract/repel the balls, e.g. a balloon rubbed on hair. (The plastic ring will not block this effect, since it's an insulator.)

There aren't that many repulsive forces out there which could be at play:

• magnetism
• electrostatic forces
• compressed material (solid, fluid or gas) pushing bodies apart

Magnetism would have worked if the bodies would not be able to rotate, or if we could produce magnetic monopoles.

Compressed springs are extremely hard to make completely invisible. Liquid or gas could be unnoticeable, but like you said, those balls don't seal the tube and cannot prevent the gas from escaping.

Which leaves you with pretty much the only option: electrostatic forces.