What does WPA/WPA2 really encrypt?

WPA (and WPA2) encrypts traffic below the level that Wireshark or similar tools capture. Those tools capture at the operating system's socket interface, not at the level of the actual network media. When you send a packet over WPA-protected WiFi, the WPA encryption isn't added until the last moment before the data is broadcast.

There might still be other encryption - for example, I could apply PGP encryption to an email and send it to SMTP server over TLS, which would be two levels of encryption... but those levels would be visible to (and, indeed, created by) the application (such as my email client). Somebody sniffing that traffic would still be able to see things like what protocol it's using (TCP, on top of IP), what port it comes from and is routing to, the destination IP address, and so on.

However, once the packet reaches the WiFi interface driver, it gets encrypted with the AES key that my machine is using for WPA. At that point, about the only things visible are the network SSID that I'm using (I think the source and destination MAC addresses may also be visible) and a vague idea of the size. Somebody without the WiFi key sniffing the network traffic using software-defined radio or a WiFi card in promiscuous mode wouldn't be able to tell the difference between my email and me sending a network ping or chatting on Skype; they wouldn't even be able to tell where the packets were going beyond the WiFi access point.

What WPA-Personal (aka WPA-PSK) does is encrypt the packets that go on the air, so that people who aren't connected to this network can't read your messages (and WEP did the same in this respect, by the way, it just did it in a different way, which suffered from a serious hole). It additionally tries to make it difficult/impossible to connect to the network without knowing the secret password.

Without this encryption (e.g. on open networks), anyone can read all the packets that are being exchanged, without even being "connected" to the network: it just needs to be close enough to "hear" the signal.

If you think of a foreign language as a kind of encryption, WPA is a bit like the situation where all machines connected to this WPA network speak their very own language language which only the AP also understands. So, machines not connected to the network can't understand anything (other than witness that some communication is taking place between the machines and the AP) and those that are connected to this network can only talk to each other by communicating via the AP.