How can I query memcached with php to get a list of all its keys in storage?

Question

I need to replicate memcached to another key value system (couchbase). How can I query the contents of a memcached server to get a list of what is in there so that I can copy it over?

https://gist.github.com/vanjos/7013411 – prayagupa Aug 20 '19 at 21:04 — prayagupa, Aug 20 '19 at 21:04

score 13 · Answer 1 · answered Jan 11 '16 at 15:12

All these solutions for Memcache so here's for Memcached

function getMemcachedKeys($host = '127.0.0.1', $port = 11211)
{

    $mem = @fsockopen($host, $port);
    if ($mem === FALSE) return -1;

    // retrieve distinct slab
    $r = @fwrite($mem, 'stats items' . chr(10));
    if ($r === FALSE) return -2;

    $slab = array();
    while (($l = @fgets($mem, 1024)) !== FALSE) {
        // sortie ?
        $l = trim($l);
        if ($l == 'END') break;

        $m = array();
        // <STAT items:22:evicted_nonzero 0>
        $r = preg_match('/^STAT\sitems\:(\d+)\:/', $l, $m);
        if ($r != 1) return -3;
        $a_slab = $m[1];

        if (!array_key_exists($a_slab, $slab)) $slab[$a_slab] = array();
    }

    // recuperer les items
    reset($slab);
    foreach ($slab AS $a_slab_key => &$a_slab) {
        $r = @fwrite($mem, 'stats cachedump ' . $a_slab_key . ' 100' . chr(10));
        if ($r === FALSE) return -4;

        while (($l = @fgets($mem, 1024)) !== FALSE) {
            // sortie ?
            $l = trim($l);
            if ($l == 'END') break;

            $m = array();
            // ITEM 42 [118 b; 1354717302 s]
            $r = preg_match('/^ITEM\s([^\s]+)\s/', $l, $m);
            if ($r != 1) return -5;
            $a_key = $m[1];

            $a_slab[] = $a_key;
        }
    }

    // close
    @fclose($mem);
    unset($mem);

    // transform it;
    $keys = array();
    reset($slab);
    foreach ($slab AS &$a_slab) {
        reset($a_slab);
        foreach ($a_slab AS &$a_key) $keys[] = $a_key;
    }
    unset($slab);

    return $keys;
}

This is the best answer currently even though people haven't voted for it yet; the question asker was asking about Memcached with a 'd'. I don't know why Mark Baker's answer got so many upvotes. — Ryan, Apr 04 '17 at 20:47
The `stats cachedump` returns INCOMPLETE data (only returns COLD items, returns at most 2MB of data, only returns first 100 items per slab in the above code), does LOCK up the whole cache while it runs, is periodically discussed to be removed from the codebase (as its for small-testbed-debugging purposes only). `lru_crawler` sounds more-promising: https://stackoverflow.com/a/60604336/2809546 — Vlad, Dec 19 '22 at 17:15

score 10 · Accepted Answer · answered Mar 22 '12 at 22:25

10

memcache >= 2.0.0

function getMemcacheKeys() {
    $memcache = new Memcache;
    $memcache->connect('127.0.0.1', 11211) 
       or die ("Could not connect to memcache server");

    $list = array();
    $allSlabs = $memcache->getExtendedStats('slabs');
    $items = $memcache->getExtendedStats('items');
    foreach($allSlabs as $server => $slabs) {
        foreach($slabs AS $slabId => $slabMeta) {
            $cdump = $memcache->getExtendedStats('cachedump',(int)$slabId);
            foreach($cdump AS $keys => $arrVal) {
                if (!is_array($arrVal)) continue;
                foreach($arrVal AS $k => $v) {                   
                    echo $k .'<br>';
                }
            }
        }
    }   
}

answered Mar 22 '12 at 22:25

Mark Baker

209,507
32
346
385

3

I just want to note here that while this works cachedump is really meant for memcached debugging purposes and isn't intended to be used like this. Future versions of memcached also might not support cachedump. (See comment 8 here from one of the core memcached contributors http://code.google.com/p/memcached/issues/detail?id=187) – mikewied Mar 25 '12 at 03:36
2

Cachedump does not seem to get all of the keys (I've been trying to do this, and then came across this link): http://code.google.com/p/memcached/wiki/NewProgrammingFAQ – jonderry Dec 19 '12 at 00:54
one small but big difference which worked for me was the **limit** parameter in `getExtendedStats` function, default value is _just_ 100. update: `$cdump = $memcache->getExtendedStats('cachedump',(int)$slabId,100000000);` – matija Jul 07 '16 at 10:00
This function is faulty as it makes the memcache readonly, at least with current version of memcache PECL (3.0.8) and memcached (1.4.21) – eithed Jul 28 '16 at 15:16
@matija you can pass 0 as the final parameter to remove the limit entirely – Gruffy Jan 20 '17 at 15:08
That's memcache, the question is for memcached. They are different libraries. – Apollo May 18 '17 at 09:36

Megas · Answer 3 · 2022-12-21T19:09:59.820

1

According to the report https://www.php.net/manual/en/memcached.getallkeys.php#123793 function Memcached::getAllKeys does not works since memcached 1.4.23. Also it no guaranty return of all keys.
Code provided by @mark-baker, @maduka-jayalath and others based on stats cachedump has limitation and does not return all keys in case you have millions of keys in memcache. Also it have performance influence on memcache server.

There is library that allows to iterate between all memcache keys asynchronously without performance influence: https://github.com/qmegas/memcache-search It even not require any memcache extension to be installed.

edited Dec 21 '22 at 19:09

answered Jun 24 '20 at 10:50

Megas

81
1
9

1

Did you mean "*`Memcached::getAllKeys` does **NOT** work*" since 1.4.23? That's what the link you reference says. – Don't Panic Dec 20 '22 at 22:45
1

@Don'tPanic Yes, you right, it's a typo. Edited my message. – Megas Dec 21 '22 at 19:10

score -1 · Answer 4 · answered Apr 30 '20 at 20:35

-1

As per the documentation:
https://www.php.net/manual/en/memcached.getallkeys.php

PECL memcached >= 2.0.0

$memcached->getAllKeys();

answered Apr 30 '20 at 20:35

WebDevNerdStuff

391
1
4
13

score -2 · Answer 5 · answered Oct 01 '13 at 17:45

Thanks for the sample code

Here is how to remove a specific key or multiple keys

I am using a helper class to remove the cache so you have to give the function a reference to the memcache connection

public static function removePriceCache(&$memcache, &$cacheAvailable) {

    if ($cacheAvailable == true) {
        $list = array();
        $allSlabs = $memcache->getExtendedStats('slabs');
        $items = $memcache->getExtendedStats('items');
        foreach ($allSlabs as $server => $slabs) {
            foreach ($slabs AS $slabId => $slabMeta) {
                $cdump = $memcache->getExtendedStats('cachedump', (int) $slabId);
                foreach ($cdump AS $keys => $arrVal) {
                    if (!is_array($arrVal))
                        continue;
                    foreach ($arrVal as $k => $v) {
                        //echo $k . '<br>';

                        if (stripos($k, "Price") !== false) {
                            $memcache->delete($k);
                        }
                    }
                }
            }
        }
    }
}

This will delete all the keys that have the word 'Price' in it..

How can I query memcached with php to get a list of all its keys in storage?

5 Answers5