Redis scan skipping keys

Question

I'm using predis (with laravel if it makes any difference) php client to work with Redis.

I need to fetch all the keys from Redis that match certain prefix and I do it like this:

$keys = [];
    foreach (new Iterator\Keyspace($this->redis(), Cache::KEY_PREFIX.'*') as $key) {
        $keys[] = $rate_key;
    }

After the work with those keys is done, operation repeats - I'm getting those keys again in again in a loop. I noticed that after a few iterations some keys are not included in $keys array.

The strangest thing is that keys that disappear never appear in next iterations. Restart of the php process (it's a daemon) fixes the problem.

I'm using Redis 3.0.2 with Predis 1.0 and PHP 5.4

P.S. Within the loop over keys, I change values for some of them. I'm not deleting any keys, however.

Answer 1

Indeed! That's because SCAN works that way, quoting from Redis documentation:

However while blocking commands like SMEMBERS are able to provide all the elements that are part of a Set in a given moment, The SCAN family of commands only offer limited guarantees about the returned elements since the collection that we incrementally iterate can change during the iteration process.

However because SCAN has very little state associated (just the cursor) it has the following drawbacks: A given element may be returned multiple times. It is up to the application to handle the case of duplicated elements, for example only using the returned elements in order to perform operations that are safe when re-applied multiple times.

So you may want to use want to use array_unique($keys) after the foreach.

To understand why the iteration works that way the best thing is to read this part of the Redis documentation.