Why are high end audio connections gold?

Gold is highly resistant to corrosion or oxidation, so prevents poor connections from those sources.It is also fairly soft, so the mating surfaces deform slightly, increasing contact area to reduce resistance. The gold plating is very thin, so the added resistance from the gold is easily overcome by its other properties.

Note that gold is only needed on the actual contact areas - gold plating (or colour) on the body of the connector is only there to attract the gullible.

Many commercial (not audiophile) connectors have selective gold plating on the contacts - the gold is only placed where it really matters.

Tarnished contacts cause distortion. You can really hear the audio quality degradation; it is not some audio-fool nonsense, like directional speaker cables or two hundred dollar amplifier power cords.

Even a dyed-in-the-wool objective measurement man like like Doug Self acknowledges this problem, in no uncertain terms, right in the middle of his treatise on "Subjectivism":

Corrosion is often blamed for subtle signal degradation at switch and connector contacts. By far the most common form of contact degradation is the formation of an insulating sulphide layer on silver contacts, derived from hydrogen sulphide air pollution; the problem seems to have become worse in recent years. This typically cuts the signal altogether, except when signal peaks temporarily punch through the sulphide layer. The effect is gross and completely inapplicable to theories of subtle degradation. Gold-plating is the only certain cure. It costs money. A switch with gold-flashed contacts can cost five times as much as the silver version. [Source] [Bold emphasis mine].

Other metals like silver will do, but the contacts will require regular polishing to perform their best. Gold is more maintenance-free.

I own a bunch of old audio and guitar gear, and I can tell you that if you put a little bit of toothpaste on a cotton swab and vigorously polish the inside of a 1/4" phone jack on some old amplifier or signal processor, it comes out black. The sound quality improves drastically.

Mainly for marketing purposes: Gold is always better than silver, right?

But also, silver oxidizes (tarnishes) readily while gold does not.