System hanging when it runs out of memory

It is possible to call OOM-killer (out of memory killer) directly by keyboard combination:

SysRq-F

SysRq key is usually combined within PrtSc key on keyboards.

OOM-killer kills some process(-es) and system becomes responsive again.

Thx Raman for advice on this feature in comments above.

PS: This helped me a lot. I agree with opinion that this is the most useful advise about that problem if it caused by Chrome or whatever memory greedy software. But you need to keep in mind that OOM-killer could kill some really important process, use it carefully.

The natural state of things is that application data is in RAM, and files are on disk.
The ideal state of things, performance wise, is that data in frequent use is in RAM, and data that isn't needed at the moment is on disk.
On a normal system, the kernel does two things to attempt to reach this ideal:

• Application data that has not been used for a while can be moved to disk: this is swap.
• Data from files that has been used recently is kept in RAM: this is the disk cache (for data read from disk) and disk buffers (for data that is about to be written to disk).

On a typical system, a significant part of the RAM is devoted to the cache and buffers (50% is a typical figure). Since RAM is a finite resource, this may require displacing some application data to swap (swap is only necessary if there is a better way to use the RAM).

On a system with no swap, there's a point when the application data is using almost all the RAM, and so there is barely any room left for cache. Then the system is likely to be slow. The kernel will not start killing applications until it really has to. As long as applications only fill 99% of the available memory, the system keeps going, but very slowly because files data has to be loaded and reloaded from disk all the time. With the same applications running, the system would be faster with swap at that point.

For more on this issue, see this lkml discussion and this blog post.

I don't know of a direct way to tell the kernel to reserve a minimum amount of RAM for the disk cache. You could set up a small part of your RAM as swap space, perhaps even compressed. There are success reports on that front, though I make no guarantees in your particular case.

Recently I found a solution to my problem.

Since the Linux OOM killer isn't able to do its job properly, I started using a userspace OOM Killer: earlyoom. It's written in C, fairly configurable and it's working like a charm for me.

I've also heard about some alternatives, like Facebook's OOMD, developed to run on their servers, but I haven't tried this one