What sort of experiment would directly test time reversal invariance?

There are numerous research groups engaged in a search for an electric dipole moment of the electron, which, if it exists, would violate time-reversal symmetry. You can see this because any dipole moment the electron might have would need to be parallel to the spin (or anti-parallel). When you reverse time, the spin necessarily flips, but the electric dipole moment would not change, so the relative orientation of the two would change. That's the best example of a T-violating phenomenon that I know of.

At the risk of unseemly self-promotion, I wrote an article on edm searches for Physics World last year. You need to register to read the whole thing, but it's free.

As an update on this old thread, the 2015 version of the Particle Data Group review on tests of conservation laws (the 2009 version of which was rightly pointed to by invisiblerhino) has an interesting update:

The BABAR experiment has reported the first direct observation of $T$ violation in the $B$ system. The measured $T$-violating parameters in the time evolution of the neutral $B$ mesons are $∆S^+_T = −1.37±0.15$ and $∆S^−_T = 1.17±0.21$, with a significance of $14σ$ [4]. This observation of $T$ violation, with exchange of initial and final states of the neutral $B$, was made possible in a $B$-factory using the Einstein-Podolsky-Rosen Entanglement of the two $B$'s produced in the decay of the $\Upsilon(4S)$ and the two time-ordered decays of the $B$'s as filtering measurements of the meson state [5].

Pointing to the reference

[4] J.P. Lees et al., Observation of Time-Reversal Violation in the $B^0$ Meson System, Phys. Rev. Lett. 109, 211801 (2012), arXiv:1207.5832.

which has pretty much what it says. For an entry-level explanation of that paper, the APS Physics Viewpoint: Particle Decays Point to an Arrow of Time is probably a good place to start. That article probably does a much better job than I could at explaining the particulars, but I'll note here that, with a $14σ$ significance, this experiment does seem to mean that

the long wait for an unequivocal time-reversal violation in particle physics is finally over.

At research level, you might be interested in the PDG review on conservation laws. Also, the review about CPT invariance gives information about tests of CPT violation in neutral kaons, at Phys. Lett. B 237, 303 (1990), Phys. Rev. D 67, 012005 (2003) and Phys. Rev. Lett. 74, 4376 (1995) for CPT violations, and at Phys. Lett. B 444, 43 (1998) and Phys. Rev. Lett. 83, 911 (1999) for CP violations.

Note that CP violation itself is still an active area of research (particularly at Belle and LHCb), since we don't know definitively how many systems display it and whether there's a deeper explanation for it.