What's the best counter for Protoss Carriers?

I'm a player who almost always goes for carriers in large games (3v3, 4v4, FFA).

As someone who uses a lot of carriers, I can tell you what I fear the most:

Terran

• Mass Vikings using focus fire. They are (way) faster than my carriers, and they have longer range. A decent number of Vikes can focus-kill a carrier in about 1 second.
• Defensively, if you support the Vikings with marines, it's even harder to deal with, since a large group of marines can very quickly kill all my Interceptors.
• Note that marines alone are terrible against carriers. You just can't catch them. Even if you kill my Interceptors, I just run away and rebuild them, or attack you somewhere else. If I've gotten to mass carriers, I probably have the resources to rebuild the Interceptors.

Protoss

• Mass Stalkers with blink. If you don't have blink, forget it, I just run away and pick off the Stalkers that chase. Even with blink, I will try to run away over gaps, but obviously, that's harder than just running away over cliffs.
• Mass (more) Carriers with focus fire.
• Forget Mass Void Rays, UNLESS they are already charged up when the battle starts. Otherwise, by the time they are charged up, half of them will be dead, and game over. Your only chance here is to get into my base and be killing my dudes so that I'm forced to come and engage you while you are charged. Even then, I will come in focus-firing, so you better be ready to react with focus-fire instantly and not waste time killing Interceptors (I see that ALL the time).

Zergling

• Infestors with Fungal Growth, Neural Parasite and Burrow, backed by mutas/hydras/corruptors
• Corruptors WITH corruption, backed by mutas or hydras.
• Corruptors alone aren't usually that effective against Carriers - I can focus fire too, and they're not too hard to run away from. Mutas alone are terrible against carriers, your bounce damage isn't very effective against the mass HP of a carrier. Mass hydras are terrible, I will always just run over a cliff or a gap and attack you somewhere else. However, if you can use fungal growth to freeze Carriers, or corruption to increase damage, and use mutas/hydras/corruptors to pick them off, then i'm in trouble.

The main problem with carriers only really arrives if you allow your opponent to build a ton of them. Alone, like Carl Clover said, they're very weak.

The secondary problem with carriers is that they're often accompanied by a mothership, which complicates matters.

One of the best ideas to counter a protoss dead-set on going carriers is to scout and provide early pressure if you see him teching quickly to a fleet beacon. He's probably spending so much money on those buildings that he has a comparatively weak army, and you should be able to do a good deal of damage.

Often the answer to a question of "how do I counter X high tier unit?" isn't so much about countering that unit once it's built, but preventing it from coming out at all.

Since nobody else has mentioned it yet...armor upgrades. The interceptor fires 2 attacks that do 5 damage each unupgraded. With a maximum of 8 interceptors per carrier, each +1 to armor reduces the total damage the carrier does by 16 per wave of interceptors. Most anti-air units will do fairly well if you spend an equal amount on them, especially if you can stay ahead in upgrades. Corrupter and vikings are especially effective.

Edit for example: Corrupters start with 2 armor, if your air armor upgrade is one ahead of the carriers' air attack upgrades then each wave of interceptors will do a mere 32 damage (80 - 16 * 3). If their attack is 1 ahead of your armor, your corrupters instead take 64 damage per wave.