kswapd0 is taking a lot of cpu
The process kswapd0 is the process that manages virtual memory. Your machine should have RAM, SWAP, and the EXT4 on your HDD/SSD. The ext4 is where everything is stored, and it is always slower to access than RAM. RAM is like a half-way running space for programs to access information quickly. Most computers have at least 4GB of RAM, which under normal conditions is plenty. When playing a game, however, you may run low on RAM space, which is where SWAP comes in.
SWAP is a fake RAM located on your HDD/SSD next to your EXT4. It is quicker to access than the EXT4, but it is much slower than actual RAM. When you run low on memory, kswapd0 moves programs that you are not using/not using as much as other programs to the SWAP, which causes extreme lag on those processes. If your game was needing 5GB RAM, 1GB at LEAST would be in SWAP. That means when it tries to access that information, it has to wait longer to get it.
This entire process causes extreme CPU usage, moving information from and to SWAP and RAM and handling the request of information all at the same time. How to solve this issue?
Tell kswapd0 to only move stuff to SWAP when you are completely OUT of RAM. This is the single most effective method to resolving SWAP issues. Run
echo vm.swappiness=0 | sudo tee -a /etc/sysctl.confwhere
0is the percent left out of100at which SWAP should be used (when you have 0% RAM left, SWAP will start taking in data). You can also just edit /etc/sysctl.conf to your liking instead of adding this command to the end of it everytime using gedit or nano or whatever, be sure to sudo though, this file is root owned. Reboot and your are set!- Reduce the consumption of RAM by other processes or close other programs while running high memory programs. This is why most games tell you to close all other windows before playing, or installations do the same. Things like file syncing services tend to take a lot of memory.
- Buy more RAM. Installing RAM is not as hard as it sounds. One or two screws on a small compartment (if you are on a laptop) and a simple click. Just be sure you are buying the correct kind!
- Lower processes of the CPU much as you did with the RAM. This will help those RAM to SWAP bursts to go much smoother.
That's the best that you can do. Others may say disable swap completely, but that is dangerous and I would NOT recommend that. That can cause entire systems to freeze up if there is a memory leak or too many applications running. Just realize that the SWAP is a failsafe for the RAM. It is definitely not as fast or efficient as RAM, but it's better than Window's Pagefile! (which accomplishes the same purpose)
EDIT: If you are interested in learning more about SWAP, see here.
kswapd0 runs at 99.9% of one CPU but is actually not swapping at all
To me it happens at times on Ubuntu 14.04 with kernel 3.19.0-50-generic (and earlier) running in a VMware vm. I have no clue, what made it appear, but it comes during idle time.
top shows:
# top
top - 09:49:35 up 5 days, 18:35, 1 user, load average: 1.00, 1.00, 0.99
Tasks: 219 total, 2 running, 217 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 25.0 sy, 0.0 ni, 74.7 id, 0.2 wa, 0.0 hi, 0.1 si, 0.0 st
KiB Mem: 3028784 total, 1874468 used, 1154316 free, 1010276 buffers
KiB Swap: 15624188 total, 3032 used, 15621156 free. 234928 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
52 root 20 0 0 0 0 R 99.7 0.0 122:15.21 kswapd0
3 root 20 0 0 0 0 S 0.3 0.0 0:29.86 ksoftirqd/0
7 root 20 0 0 0 0 S 0.3 0.0 9:49.47 rcu_sched
Temporary solution
a reboot solved the problem - temporarily.
following the answer on serverfault (kswapd often uses 100% CPU when swap is in use) there where the same settings on my system:
# cat /proc/sys/vm/swappiness
60
# cat /proc/sys/vm/vfs_cache_pressure
100
# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
the solution was actually running echo 1 > /proc/sys/vm/drop_caches as root user:
# cat /proc/sys/vm/drop_caches
0
# echo 1 > /proc/sys/vm/drop_caches
# cat /proc/sys/vm/drop_caches
1
Or, as pointed out by theTuxRacer (thanks!), use the following command in case you are not logged in as root:
echo 1 | sudo tee /proc/sys/vm/drop_caches
now it's fine:
# top
top - 10:08:58 up 5 days, 18:55, 1 user, load average: 0.72, 0.95, 0.98
Tasks: 220 total, 1 running, 219 sleeping, 0 stopped, 0 zombie
%Cpu(s): 0.0 us, 0.2 sy, 0.0 ni, 99.8 id, 0.0 wa, 0.0 hi, 0.0 si, 0.0 st
KiB Mem: 3028784 total, 681704 used, 2347080 free, 2916 buffers
KiB Swap: 15624188 total, 3032 used, 15621156 free. 81924 cached Mem
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
9 root 20 0 0 0 0 S 0.3 0.0 14:10.40 rcuos/0
1 root 20 0 45652 8124 2888 S 0.0 0.3 1:54.98 init
Permanent solution (to be found)?
but since the actual reason is not yet known, and I did not fine any suitable explanation on the net, this is not a permanent solution. Actually, the selected answer could be the permanent solution. I just wanted to add this for future reference, as a reboot (to make sysctl take effect) is not always possible.
An other solution might be to set THP to either madvice or never (see poige's comment to his answer, How do I modify “/sys/kernel/mm/transparent_hugepage/enabled” and the referenced MongoDB Manual on Disable Transparent Huge Pages (THP))
cron job
i've set up the following batch as a cron job as a "permanent" solution:
#!/bin/bash
# Rev 2: Use ps instead of top
## run as cron, thus no $PATH, thus need to define all absolute paths
cpu=$(/usr/bin/printf %.0f $(/bin/ps -o pcpu= -C kswapd0))
[[ -n $cpu ]] \
&& (( $cpu >= 90 )) \
&& echo 1 > /proc/sys/vm/drop_caches \
&& echo "$$ $0: cache dropped (kswapd0 %CPU=$cpu)" >&2 \
&& exit 1
exit 0
invoked from root@localhost:~# crontab -e with
# m h dom mon dow command
* * * * * /bin/bash /path/to/batch/drop_caches.sh >> /var/log/syslog 2>&1
Note: Above cron job script has been adapted to include Fredrik Erlandsson's suggestion: A much simpler and more efficient way to determine kswapd0's CPU usage. Thanks!
16.0414.04swap