Making Linux read swap back into memory

It might help to up /proc/sys/vm/page-cluster (default: 3).

From the kernel documentation (sysctl/vm.txt):

page-cluster

page-cluster controls the number of pages up to which consecutive pages are read in from swap in a single attempt. This is the swap counterpart to page cache readahead. The mentioned consecutivity is not in terms of virtual/physical addresses, but consecutive on swap space - that means they were swapped out together.

It is a logarithmic value - setting it to zero means "1 page", setting it to 1 means "2 pages", setting it to 2 means "4 pages", etc. Zero disables swap readahead completely.

The default value is three (eight pages at a time). There may be some small benefits in tuning this to a different value if your workload is swap-intensive.

Lower values mean lower latencies for initial faults, but at the same time extra faults and I/O delays for following faults if they would have been part of that consecutive pages readahead would have brought in.

The documentation doesn't mention a limit, so possibly you could set this absurdly high to make all of swap be read back in really soon. And of course turn it back to a sane value afterwards.

It seems to me that you can't magically "make the system responsive again". You either incur the penalty or reading pages back from swap space into memory now or you incur it later, but one way or the other you incur it. Indeed, if you do something like swapoff -a && swapon -a then you may feel more pain rather than less, because you force some pages to be copied back into memory that would otherwise have never been needed again and eventually dropped without being read (think: you quit an application while much of its heap is swapped out; those pages can be discarded altogether without ever getting read back in to memory).

but this clears the pages from swap, so they need to be written again the next time I run the script.

Well, pretty much any page that gets copied back from swap into main memory is about to be modified anyway, so if it ever needed to be moved back out to swap again in the future, it would have to be written anew in swap anyway. Keep in mind that swap is mainly heap memory, not read-only pages (which are usually file-backed).

I think your swapoff -a && swapon -a trick is as good as anything you could come up with.

You may try adding the programs you most care about to a cgroup and tuning swappiness so that the next time the application runs the programs you add are less likely to be candidates for swapping.

Some of their pages will likely still be swapped out but it may get around your performance problems. A large part of it is probably just the "stop and start" behavior when a lot of a program's pages are in swap and the program has to continually pause in order swap its pages into RAM but only in 4k increments.

Alternatively, you may add the application that's running to a cgroup and tune swappiness so that the application is the one that tends to use the swap file most. It'll slow down the application but it'll spare the rest of the system.