Why do higher frequency waves have better penetration?

It is not true that higher frequencies always penetrate further than lower ones. The graph of transparency of various materials as a function of wavelength can be quite lumpy. Think of colored filters, and those only apply to a narrow octave of wavelengths we call visible light.

What you are apparently thinking of is wavelengths so short that the energy is very high, like xrays and gamma rays. These go thru things solely because of their high energy. At lower energies (longer wavelengths), the waves interact with the material in various ways so that they can get absorbed, refracted, reflected, and re-emitted. These effect vary in non-monotonic ways as a function of wavelength, the depth of the material, it's resistivity, density, and other properties.

The main advantage of higher frequencies is that they require shorter antennas for decent reception quality, and that's important for mobile devices. They also allow a wider band for modulating signals, so you can obtain higher frequency transmission.

But high frequencies are more sensitive to reflection, so they will have a harder time passing through walls and obstacles in general. At the same time, they will more easily leak through holes: a rule of thumb is that if you have a hole of the size of the wavelength, the signal can leak through it. But at the same time, you can't rely on it for a good transmission: so I'd say that the limit is quite fuzzy.

For further insights, look at line-of-sight propagation: microwave frequency can be refracted by smaller object than lower radio frequency, as it's strongly dependent on the wavelength. The comparison arises by the fact that microwaves have a spectrum that is more similar to the optical wavelengths, so they will suffer from some of the phenomena that hold for optics.

In fact higher frequencies have worse penetration capabilities. If you consider a purely theoretical model, the so-called skin depth, which gives the thickness of the layer of a conductor to which an electromagnetic wave of a given frequency is able to penetrate it, you will see that the skin depth is inversely proportional with the square root of the frequency:

\$ \delta = \sqrt{\frac{2\rho}{\omega\mu}}\$

(\$\rho\$ is the resistivity, \$\mu\$ the magnetic permeability of the material).

This has also as a consequence that AC currents do not use the whole cross-section of a wire (and a properly designed hollow one would do the same job) and that's (partly) why a smaller antenna will do for proper transmission.

But in reality things are much more complicated than that. Wireless HD video is serious engineering challenge (partly) because the high frequency signals necessary to provide the appropriate bandwidth tend to bounce off the walls. At really high frequencies (i.e. ~60 GHz) necessary for such applications other absorption/reflection phenomena can compromise transmission: e.g. absorption by oxygen (in the air). This depends very much on the medium through which your wave needs to go through.

So, the short answer is no, higher frequencies aren't able to go better through walls than low frequencies.