Atiyah-Patodi-Singer Eta invariant and Chern-Simons form
1) The eta invariant itself depends on the metric, but the relative eta invariant is in many cases (see comments) a homotopy invariant. The relative eta invariant is defined to be the difference of the eta invariants associated to the Dirac operator twisted by two different flat Hermitian bundles (i.e. unitary representations of the fundamental group).
2) The relation between the eta invariant and Chern-Simons invariants is a little bit subtle, but it is explained in detail in section 4 of "Spectral Asymmetry and Riemannian Geometry II" by A-P-S.
3) Arguably the most important examples are lens spaces - this is how it was first realized that the defect in the signature theorem for manifolds with boundary is non-local, for example (if it were local it would be multiplicative for coverings). There is also an interesting paper called "Eta Invariants, Signature Defects of Cusps, and Values of L-Functions" by Atiyah, Donnelly, and Singer in which the eta invariant associated to the signature operator on a Hilbert modular variety with the cusps chopped off is calculated in terms of values of Shimazu L-functions. This was apparently one of the motivating examples for the theory of eta invariants, but I don't know what actual arithmetic significance it has.
3) In how many non-trivial cases the eta invariant is explicitly calculable?
I have computed the eta invariants for the $spin^c$ Dirac operators on Seifert $3$-manifolds.
See this paper for the special case of circle bundles. Here I describe in some detail how one goes about computing eta invariants (never easy) and I included some references about computations of the eta invariant that arises in the APS problem for the signature operator. For the more general case of Seifert manifolds see this paper.
The lens spaces mentioned by Paul Siegel are special cases of Seifert manifolds.
Ad (3). Computing $\eta$-invariants on the nose is notoriously difficult. However, sometimes one needs $\eta$-invariants as ingredients of other differential topological invariants ($\rho$-invariants as in Pauls answer are an example. The Eells-Kuiper invariant is another one of a slightly different flavour). In this case, one can often compute $\eta$-invariants up to some correction terms, which are more accessible. Let me sketch a few things that work sometimes. Most, but not all of the following is explained in this overview article. Most of the following is applicable to 3-manifolds.
Direct computation from the spectrum of the operator. This works in very few cases where one does not expect $\eta(D)=0$ from the very beginning. The first problem is that one needs to know the spectrum. Hitchin did this for Berger spheres in his phd thesis. Somewhat related are the papers by Millson and Moscovici-Stanton that relate $\eta$-invariants of locally symmetric spaces (including hyperbolic 3-manifolds) to $\zeta$-functions associated with the geodesic flow.
Using the Atiyah-Patodi-Singer theorem. Try to find an explicit cobordism to a manifold where the $\eta$-invariant is known, and apply APS. Kruggel used this method to compute Kreck-Stolz invariants of Eschenburg spaces.
If $M$ admits a finite normal covering $M=\tilde M/\Gamma$ and the $\Gamma$-equivariant $\eta$-invariant of $\tilde M$ is known, one can recover $\eta$- and $\rho$-invariants of $M$. The computation of $\eta$-invariants for lense spaces is a combination of (2) and (3).
Find a similar operator $\tilde D$ on the same vector bundle, such that $\eta(\tilde D)$ is computable, then try to access the difference. This has been done by Deninger and Singhof for Heisenberg manifolds, or here for the cubical Dirac operator on quotients of compact Lie groups.
If the manifold $M$ happens to be a fibre bundle over a base $B$, one can use the adiabatic limit technique of Bismut-Cheeger and Dai. This also works for Seifert fibrations, see also Liviu's answer. However, one needs to know the $\eta$-forms of a family of fibrewise operators, and one has to assume that their kernels form a vector bundle over $B$.
If $M=M_-\cup_{M_0}M_+$ with $M_0$ totally geodesic, one can use a gluing theorem, see Kirk-Lesch. Now one needs to know the $\eta$-invariants of the pieces with respect to suitable boundary conditions, and a contribution from $M_0$ relating those boundary conditions. This technique has been applied here.
I am happy to add to this list, just leave a comment.