An interesting topological space with $4$ elements

Quoting the actual answer compiled by the asker from comments to remove the question from unanswered pool.

According to Miha's comment below (which is an answer), $X$ is called the pseudocircle, and the definite source for the general phenomenon is:

Singular homology groups and homotopy groups of finite topological spaces, by Michael C. McCord, Duke Math. J., 33(1966), 465-474, doi:10.1215/S0012-7094-66-03352-7.

The pseudocircle already appeared a couple of times on math.SE, for example in question/56500.

The natural relationship between $S^1$ and $X$ arises as follows. Consider $X$ as a preordered set in its specialization order: explicitly, the order is $\eta,\eta'\leq x,x'$ (or the reverse of that depending on your conventions). We can then take the nerve $N(X)$: explicitly, $N(X)$ is a simplicial set in which an $n$-simplex is an order-preserving map $[n]\to X$. The nondegenerate simplices are the injective order-preserving maps, i.e. the chains in the poset $X$. There are four nondegenerate $0$-simplices (the $4$ points of $x$) and $4$ nondegenerate $1$-chains (the $4$ chains in $X$ of size $2$), which connect the points of $X$ cyclically: $\eta\leq x$, then $x\geq \eta'$, then $\eta'\leq x'$, then $x'\geq \eta'$.

This means that the geometric realization $|N(X)|$ is homeomorphic to a circle (it's just a cyclic graph with 4 vertices). Moreover, there is a canonical continuous map $p:|N(X)|\to X$, which sends a point that is in the interior of a nondegenerate simplex $\sigma$ (i.e., a chain in $X$) to its least element in $X$. Explicitly, this maps the four vertices of $|N(X)|$ to the corresponding points of $X$, and the interiors of the edges to either $\eta$ or $\eta'$ depending on which of the two they have as a vertex.

Now for the magical fact: this canonical map $p:|N(X)|\to X$ is a weak homotopy equivalence, so $X$ is weak homotopy equivalent to $S^1$. And even more magically, this generalizes. In particular, McCord proved the following results in his paper Singular homology groups and homotopy groups of finite topological spaces:

Theorem: Let $P$ be any preordered set, considered as a topological space by letting the open sets be the downward closed sets (i.e., the finest topology with $P$ as its specialization order). Then the natural map $|N(P)|\to P$ constructed as above is a weak homotopy equivalence.

Corollary: Let $X$ be any finite topological space and equip it with its specialization order. Then the natural map $|N(X)|\to X$ is a weak homotopy equivalence.

Proof: A topology on a finite set is determined by its specialization order, so the topology on $X$ is the same as the topology induced by the specialization order as in the Theorem.

Corollary: Every finite CW-complex is weak homotopy equivalent to a finite topological space.

Proof: Let $X$ be a finite CW-complex. Then $X$ is homotopy equivalent to the geometric realization of some finite simplicial complex $Y$. Let $P$ be the poset of faces of $Y$, ordered by inclusion. Then $N(P)$ is just the barycentric subdivision of $Y$, and in particular there is a natural homeomorphism $|N(P)|\to |Y|$. Thus by the Theorem, $X$ is weak homotopy equivalent to $P$.

Note in particular that these results show that the relationship between your space $X$ and $S^1$ is "one-sided": $S^1$ is naturally associated to $X$, but $X$ is not naturally associated to $S^1$. Indeed, there are many different "finite models" of $S^1$; for instance, the poset of faces in any triangulation of $S^1$ is a "finite model" of $S^1$ as in the proof of the second Corollary.