Reducing garbage-collection pause time in a Haskell program

As mentioned in other answers, the garbage collector in GHC traverses live data, which means the more long-living data you store in memory, the longer GC pauses will be.

GHC 8.2

To overcome this problem partially, a feature called compact regions was introduced in GHC-8.2. It is both a feature of the GHC runtime system and a library that exposes convenient interface to work with. The compact regions feature allows putting your data into a separate place in memory and GC won't traverse it during the garbage collecting phase. So if you have a large structure you want to keep in memory, consider using compact regions. However, the compact region itself doesn't have mini garbage collector inside, it works better for append-only data structures, not something like HashMap where you also want to delete stuff. Though you can overcome this problem. For details refer to the following blog post:

• Using compact regions (by Alexander Vershilov)

GHC 8.10

Moreover, since GHC-8.10 a new low-latency incremental garbage collector algorithm is implemented. It's an alternative GC algorithm which is not enabled by default but you can opt-in to it if you want. So you can switch the default GC to a newer one to automatically get features provided by compact regions without the need to do manual wrapping and unwrapping. However, the new GC is not a silver bullet and doesn't solve all the problems automagically, and it has its trade-offs. For benchmarks of the new GC refer to the following GitHub repository:

• Low-latency incremental GC benchmarks

You're actually doing pretty well to have a 51ms pause time with over 200Mb of live data. The system I work on has a larger max pause time with half that amount of live data.

Your assumption is correct, the major GC pause time is directly proportional to the amount of live data, and unfortunately there's no way around that with GHC as it stands. We experimented with incremental GC in the past, but it was a research project and didn't reach the level of maturity needed to fold it into the released GHC.

One thing that we're hoping will help with this in the future is compact regions: https://phabricator.haskell.org/D1264. It's a kind of manual memory management where you compact a structure in the heap, and the GC doesn't have to traverse it. It works best for long-lived data, but perhaps it will be good enough to use for individual messages in your setting. We're aiming to have it in GHC 8.2.0.

If you're in a distributed setting and have a load-balancer of some kind there are tricks you can play to avoid taking the pause hit, you basically make sure that the load-balancer doesn't send requests to machines that are about to do a major GC, and of course make sure that the machine still completes the GC even though it isn't getting requests.