Differences in Homology/Cohomology of a CW Pair (X,A) when A is empty versus a point

Hatcher's definition of a "good pair" $(X,A)$ requires $A$ to be nonempty (see page 114). Indeed, the theorem that the homology of $(X,A)$ is isomorphic to the reduced homology of $X/A$ is not true when $A=\emptyset$. Following Hatcher's Proposition 2.22, we do still have an isomorphism $H_n(X,A)\cong H_n(X/A,A/A)$, but we don't have $H_n(X/A,A/A)\cong \tilde{H}_n(X/A)$ because $A/A$ is empty rather than being a single point.

(Actually, there is a way to make all the theorems still work when $A$ is empty, but it requires redefining $X/A$. The correct definition of $X/A$ when $A$ could be empty is the quotient of $X\sqcup\{*\}$ that identifies every point of $A$ with newly adjoined point $*$. When $A$ is nonempty this is the same as the usual quotient of $X$ that identifies all points of $A$ together, but when $A$ is empty it gives $X\sqcup\{*\}$ where the empty set $A$ has been turned into a new point. What's really going on is that the operation sending $(X,A)$ to $X/A$ is more natural when $X$ is a pointed space, and so if $X$ is just a space you should freely turn it into a pointed space before forming $X/A$. Or, for a better explanation, see Paul Frost's answer.)

The relative homology of $(X,\varnothing)$ is just that of $X$. But the relative homology of $(X,\ast)$ is the reduced homology, since it comes from the exact sequence

$$0\to C(\ast)\to C(X) \to C(X,\ast)\to 0$$

and $C(\ast)$ is the complex where $\mathbb Z$ is concentrated in degree $0$.