What's the simplest way to get a dump of all memcached keys into a file?

disclaimer: I don't know what I"m doing, just sounded like an interesting problem.

Did you see this article? "How to Dump Keys from Memcache" by Lars Windolf.

From the article:

Memcache itself provide the means to peak into the data. The protocol provides commands to peak into the data that is organized by slabs (categories of data of a given size range. There are some significant limitations though:

  • You can only dump keys per slab class (keys with roughly the same content size)
  • You can only dump one page per slab class (1MB of data)
  • This is an unofficial feature that might be removed anytime.

Effectively, it requires some knowledge of how memcache stores data in memory (which I don't). You need to find each 'slab', then you can dump the keys for that slab, and then ultimately, the values for those keys.

There is a tools section in the article which uses various languages to dump at least the keys, but only the perl script dumps both keys and values.


memccat

Here is the script which I'm using to dump all the objects into corresponding files:

while read -r key; do
    [ -f "$key" ] || echo "get $key" | nc localhost 11211 > "$key.dump";
done < <(memcdump --server localhost)

It uses memcdump command which should be part of memcached utils.

For compressed objects, see: How to dump a compressed object for given key from Memcache?

memcdump

To dump a list of keys from a server, use memcdump/memdump tool, e.g.

memcdump --servers=localhost | tee my_keys.lst

To print the value of one item, use netcat:

echo "get 13456_-cache-some_object" | nc localhost 11211

To dump all objects into the screen via memcdump/memdump and netcat:

memcdump --servers=localhost | xargs -L1 -I% sh -c 'echo "get %" | nc localhost 11211'

memcached-tool

In the recent version of memcached there is also memcached-tool command, e.g.

memcached-tool localhost:11211 dump | less # dumps keys and values

There is a hardcoded limit of 2MB for the dump of a slab. Unless you rewrite the do_item_cachedump, you will not be able to get all keys out.

Tags:

Memcached

Recent Posts