Why is there a duality between spaces and commutative algebras?
I don't claim to have a complete answer but here are some miscellaneous comments.
Note that topological spaces are already very nearly defined to be dual to certain commutative algebra-like structures, namely their frames of open subsets. The simplest interesting case of this duality is a duality between finite sets and finite Boolean rings / algebras. One way to think about this conceptually is that the open subsets of a topological space axiomatize verifiable or semidecidable properties of a point in that space: that is, properties such that if they hold you can check that that's true, but such that if they don't hold you can't necessarily check that that's true. See this math.SE question for more discussion on this point. For example, the open sets in the product topology on $\{ 0, 1 \}^{\mathbb{N}}$ correspond precisely to those properties of an infinite sequence of zeroes and ones that you can verify by looking at finitely many terms of the sequence. (Once we agree that verifiable properties are a cool thing to look at, we should also agree that we care about logical operations on them, like AND and OR, and this is where the commutative algebra-like structure comes from.)
One way to define the category of commutative $k$-algebras is as follows. Let $\text{Poly}(k)$ be the category whose objects can be thought of as the affine spaces $\mathbb{A}^n$ over $k$ and whose morphisms $\mathbb{A}^n \to \mathbb{A}^m$ are $m$-tuples of polynomials in $n$ variables over $k$, with composition given by composition of polynomials. On the one hand, this is a Lawvere theory, and the category of commutative algebras over $k$ can be defined as the category of models of it, or more explicitly as the category of product-preserving functors $\text{Poly}(k) \to \text{Set}$. On the other hand, this is a full subcategory of the category of varieties over $k$, and one can try to probe varieties by mapping affine spaces into them; this turns a variety into a presheaf $\text{Poly}(k)^{op} \to \text{Set}$. There is a general relationship between functors and presheaves on the same category called Isbell duality, and the nLab suggests that this is the general setting for adjunctions between things that look like spaces and things that look like commutative algebras, although I haven't really internalized this.
The two points made above can be related as follows. One way to think about the category of sets is that it is the category of ind-objects of the category of finite sets; equivalently, it's the category of presheaves $\text{FinSet}^{op} \to \text{Set}$ sending finite colimits to finite limits. Now, $\text{FinSet}^{op}$ is the category of finite Boolean rings, and the category of Boolean rings can be thought of as the category of ind-objects in this; equivalently, it's the category of functors $\text{FinSet} \to \text{Set}$ preserving finite limits. These two descriptions should be related by Isbell duality.
Why is $\text{FinSet}$ so important, anyway? Well, one categorical property that categories of spaces often share that isn't true in general is that binary products tend to distribute over binary coproducts. This is true in particular in any cartesian closed category, such as $\text{Set}$, and any category with this property behaves in some sense like a categorified commutative ring (!). It turns out that $\text{FinSet}$ is the free distibutive category on a point.
An addendum to my comment to Qiaochu's answer, which amounts to expanding on Qiaochu's answer more generally: If we just add splitting of idempotents to our requirements on $(\mathcal{C}, \otimes)$, then $\mathrm{Comm}(\mathcal{C})$ is co-extensive! That is, the following equivalent conditions on a symmetric monoidal category $(\mathcal{C},\otimes)$ guarantee that $\mathrm{Comm}(\mathcal{C})$ is co-extensive:
$\mathcal{C}$ has finite biproducts and split idempotents, and $\otimes$ preserves finite biproducts in each variable separately.
$\mathcal{C}$ is enriched in commutative monoids and is Cauchy-complete and symmetric monoidal category in the enriched sense.
I find these conditions to be surprisingly mild and conceptual, compared to the definition of an extensive category, which I find a bit "fussy". I think these are pretty natural and minimal requirements of a category $\mathcal{C}$ to think of it as "a category of commutative modules", and being of the form $\mathrm{Comm}(\mathcal{C})$ for such a $\mathcal{C}$ is a pretty natural requirement for a category to be considered "a category of commutative algebras" -- although admittedly this is probably not the most natural way to think about commutative $C^\ast$-algebras. So the upshot is that any category of commutative algebras is dual to an extensive category, which is a good start toward being "a category of spaces", whatever that means.
But I don't know about the reverse direction -- given a "category of spaces", how likely is it to be dual to a category of algebras? This seems tricky, because there are certainly categories of spaces, such as projective varieties, which don't seem to be dual to categories of algebras -- we need to formulate some kind of "affineness criterion" in order to have a shot.
Anyway, it is not hard to show that $\mathrm{Comm}(\mathcal{C})$ is co-extensive when $\mathcal{C}$ is as above (it hardly can be, given how "clean" the hypotheses are!):
- The 0-ary case of co-extensivity says that we have a strict terminal object (it admits no maps out except isomorphisms), which is true because the terminal object in $\mathrm{Comm}(\mathcal{C})$ is $0$, and if $0 \to A$ is a ring homomorphism, then the unit of $A$ is $0$ and the identity on $A$ factors through $0$.
- The binary case says that the pushouts of the legs of a product decomposition again form a product decomposition. The product in $\mathrm{Comm}(\mathcal{C})$ is the biproduct in $\mathcal{C}$, so a map $A \times B \to C$ is induced by maps $f: A \to C$ and $g: B \to C$ in $\mathcal{C}$. Then composing $f$ and $g$ with the units on $A$ and $B$ we get mutually annihilating idempotents on $C$ which span $C$, and splitting the idempotents, we get a biproduct decomposition of $C$ in $\mathcal{C}$, which is a product decomposition in $\mathrm{Comm}(\mathcal{C})$.
All these five equivalences $\mathcal{C}^{op} \simeq \mathcal{D}$ follow the following pattern: There is an object $P$ which, in some sense, belongs to both $\mathcal{C}$ and $\mathcal{D}$. More precisely, there are objects $P_{\mathcal{C}} \in \mathcal{C}$ and $P_{\mathcal{D}} \in \mathcal{D}$ which are "similar" in the sense that for example they share the same underlying set, maybe more. Moreover, the hom-functor $\hom(-,P_{\mathcal{C}})$ lifts to a functor $$\underline{\hom}(-,P_{\mathcal{C}}) : \mathcal{C}^{op} \to \mathcal{D},$$ and likewise $\hom(-,P_{\mathcal{D}})$ to a functor $$\underline{\hom}(-,P_{\mathcal{D}}) : \mathcal{D} \to \mathcal{C}^{op}.$$ Then in most examples it is rather easy to construct an adjunction between these functors, since unit and counit may be defined as "evaluation" $$X \to \underline{\hom}(\underline{\hom}(X,P),P),\,x \mapsto (f \mapsto f(x)).$$ For more details, see the nlab article on dualizing objects. This is the purely formal part. The nontrivial part is to determine the fixed objects of this adjunction. If we are lucky, every object is a fixed object, so that $\mathcal{C}^{op} \simeq \mathcal{D}$. But often, we have to shrink $\mathcal{C}$ and $\mathcal{D}$ further.
In Gelfand duality (2), $P=\mathbb{C}$ "is" both an object of $\mathsf{Top}$ and of $*\mathsf{Alg}$ (unital $^*$-algebras), and the Gelfand spectrum $$\underline{\hom}_{*\mathsf{Alg}}(-,\mathbb{C}) : *\mathsf{Alg} \to \mathsf{Top}^{op} $$ is left adjoint to the function algebra $$\underline{\hom}_{\mathsf{Top}}(-,\mathbb{C}) : \mathsf{Top}^{op} \to *\mathsf{Alg}.$$ The fixed objects are (at least) the unital commutative $C^*$-algebras resp. the compact Hausdorff spaces, so that these categories are anti-equivalent. Similarly, in algebraic geometry (1), $\mathbb{C}$ "is" both a variety and an algebra, and in Stone duality (3), $\{0,1\}$ "is" both a space and a Boolean algebra. I am not familiar with the other examples (4), (5), but here the dualizing objects seem to $\mathbb{Q}$ and $\overline{\mathbb{F}_p}$.