Why does linux out-of-memory (OOM) killer not run automatically, but works upon sysrq-key?

The reason the OOM-killer is not automatically called is, because the system, albeit completely slowed down and unresponsive already when close to out-of-memoryy, has not actually reached the out-of-memory situation.

Oversimplified the almost full ram contains 3 type of data:

1. kernel data, that is essential
2. pages of essential process data (e.g. any data the process created in ram only)
3. pages of non-essential process data (e.g. data such as the code of executables, for which there is a copy on disk/ in the filesystem, and which while being currently mapped to memory could be "reread" from disk upon usage)

In a memory starved situation the linux kernel as far as I can tell it is kswapd0 kernel thread, to prevent data loss and functionality loss, cannot throw away 1. and 2. , but is at liberty to at least temporarily remove those mapped-into-memory-files data from ram that is form processes that are not currently running.

While this is behaviour which involves disk-thrashing, to constantly throw away data and reread it from disk, can be seen as helpful as it avoids, or at least postpones the necessariry removing/killing of a process and the freeing-but-also-loosing of its memory, it has a high price: performance.

[load pages from disk to ram with code of executable of process 1]
[ run process 1 ] 
[evict pages with binary of process 1 from ram]
[load pages from disk to ram with code of executable of process 2]
[ run process 2 ] 
[evict pages with binary of process 2 from ram]
[load pages from disk to ram with code of executable of process 3]
[ run process 3 ] 
[evict pages with binary of process 3 from ram]
....
[load pages from disk to ram with code of executable of process 1]
[ run process 1 ] 
[evict pages with binary of process 1 from ram]

is clearly IO expensive and the system is likely to become unresponsive, event though technically it has not yet run out completely of memory.

From a user persepective however it seems, to be hung/frozen and the resulting unresponsive UI might not be really preferable, over simply killing the process (e.g. of a browser tab, whose memory usage might have very well been the root cause/culprit to begin with.)

This is where as the question indicated the Magic SysRq key trigger to start the OOM manually seems great, as the Magic SysRq is less impacted by the unresponsiveness of the system.

While there might be use-cases where it is important to preserve the processes at all (performance) costs, for a desktop, it is likely that uses would prefere the OOM-killer over the frozen UI. There is patch that claims to exempt clean mapped fs backed files from memory in such situation in this answer on stackoverflow.

You could watch the file /sys/fs/cgroup/memory/memory.oom_control, durring a stress test.

or

You could look at it's last modified date, to see if it was changed around the time of the last lockup. This will tell you if it was even atempting to do it's job.

under_oom 0

That is your issue:

under_oom    0 or 1 (if 1, the memory cgroup is under OOM, tasks may
             be stopped.)

If set to 1, it means it's under oom control. Enabled.
If set to 0, like that it's not under oom control. Disabled.