When to turn off Transparent Huge Pages for redis

It is rather annoying that searching for "transparent huge pages" yields top results about how to disable them because Redis and some other databases cannot handle transparent huge pages without performance degradation.

These applications should do any of:

• Use prctl(PR_SET_THP_DISABLE, ...) call to opt out from using transparent huge pages.
• Be started by a script which does this call for them prior to fork/exec the database process. PR_SET_THP_DISABLE get inherited by child processes/threads for exactly this scenario when an existing application cannot be modified.

prctl(PR_SET_THP_DISABLE, ...) has been available since Linux 3.15 circa 2014, so that there is little excuse for those databases to not mention this solution, instead of giving this poor/panic advice to their users to disable transparent huge pages for the entire system.

3 years after this question was asked, Redis got disable-thp config option to make prctl(PR_SET_THP_DISABLE, ...) call on its own, by default.

My production memory-intensive processes go 5-15% faster with /sys/kernel/mm/transparent_hugepage/enabled set to always, and this is why I cannot appreciate those search results for "transparent huge pages" spammed with Redis advise to disable them. That advise is poor.

Turning off transparent hugepages is a bad idea, and redis no longer recommends it.

What you should do instead is make sure transparent_hugepage is not set to always. (This is what recent versions of redis check for.) You can check the current value of the setting with:

$ cat /sys/kernel/mm/transparent_hugepage/enabled

And correct it like so:

# echo madvise >/sys/kernel/mm/transparent_hugepage/enabled

Although no action is likely to be necessary, since madvise is typically the default setting in recent linux distros.

Some background:

• transparent_hugepage = always: can force applications to use hugepages unless they opt out with madvise. This has a lot of problems and is rarely enabled.
• transparent_hugepage = never: does not fulfill allocations with hugepages, even if the application requests it with madvise
• transparent_hugepage = madvise: allows applications to opt-in to hugepages. This is normally a good idea because hugepages can improve performance in some applications, but this setting doesn't force them on applications that, like redis, don't opt in

Turn it off. The problem lies in how THP shifts memory around to try and keep or create contiguous pages. Some applications can tolerate this, most databases cannot and it causes intermittent performance problems, some pretty bad. This is not unique to Redis by any means.

For your application, especially if it is JAVA, set up real HugePages and leave the transparent variety out of it. If you do that just make sure you alocate memory correctly for the app and redis. Though I have to say, I probably would not recommend running both the app and redis on the same instance/server/vm.