How are topological invariants constructed?

A topological invariant is a continuous map $n$: $$ \mathfrak{H}\ni H\mapsto n\left(H\right)\in S $$ where $H$ is the Hamiltonian of your system and $S$ is some topological space. $\mathfrak{H}$ is the space of all admissible Hamiltonians.

Unfortunately the exact definition of $\mathfrak{H}$ is still a matter of current research. To keep things short, we do know a few things about what should characterize its elements: they should have some sort of gap (spectral gap or mobility gap) which makes it into an insulator in case the system under consideration is infinite in all space directions (half-infinite, edge systems, are allowed to be conductors, but should descend from gapped infinite systems). They should be local (have off-diagonal matrix-elements which decay exponentially in the distance between the two points in space of the matrix element). They should be self-adjoint as all Hamiltonians. They should possibly obey certain symmetries, meaning commute or anti-commute with some (fixed) unitary or anti-unitary symmetry operator.

So far the only topological invariants constructed were such $S$ is equal to $\mathbb{Z}$ or $\mathbb{Z}_2$. But this is not set in stone. The result of having $S$ discrete is that continuous maps into it are locally constant, hence the name invariant.

How is such an $n$ constructed?

1) Come up with a map $n$, and prove it is continuous. If I am not mistaken, this is how the FKM invariant came to be.

2) Use a mathematical theory of (homotopy) classification of spaces to systematically generate invariants. If you further assume your system is translation invariant (a physically poor decision, because disorder is necessary to explain key features of the IQHE for example) then you could view $\mathfrak{H}$ as a space of vector bundles and then use the theory of classification of vector bundles. See the book by Milnor called "Characteristic Classes" for an introduction. The first Chern number is an example of an invariant which was constructed in this way in the seminal paper of TKKN. If you don't assume translation invariance, you enter the realm of non-commutative geometry and then there is a parallel theory of characteristic classes developed by Alain Connes. Here the mathematical object replacing vector bundles are projections in C-star algebras. Their homotopy theory is called "C-star algebra K-theory" and a good book to start with is Rordam's. Then the parallel of Milnor's book in this setting would be Higson's book called "Analytic K-Homology". Now there is a formula for the non-commutative first Chern number, which does not require Bloch decomposition (just an example which complements the example of the first Chern number). This view was championed by Jean Bellissard. Whether in commutative or non-commutative geometry, the construction of such invariants is entirely systematic and leaves no room for choice. Note however that this classification is not complete so further invariants may be constructed beyond the framework of (commutative or non-commutative) characteristic classes.

As far as I understand things, it was only in the IQHE where first the physical quantity was calculated and then later was "recognized" as topological object. I think all other invariants so far have been constructed the other way around. That isn't to say they don't have physically measurable meaning. Another interesting question is whether these quantities represent response of the system to a driving force (as in the IQHE case, where the Hall conductivity is a linear response to driving the system with an electric field, which is computed via Kubo's formula). The answer to this question seems to be "no", and the FKM index is probably one counter-example.

This paper A short guide to topological terms in the effective theories of condensed matter by Akihiro Tanaka and Shintaro Takayoshi (Feb 2015) surveying the topological terms in the effective field theories of CMP may be what you want to start with. However the review might not exhaust all that have appeared in the literature.