The method of iterating over all the key in Redis

Sometimes we need to get a list of keys with a particular prefix from Redis to manually process the data, either by changing its value or by deleting the key. Here’s a question: How do we find a list of keys that satisfy a particular prefix from the vast number of keys in Redis?

Redis provides us a simple command: keys, to list all keys that match regular expression.

127.0.0.1:6379> keys *y11*
1) "key119"
2) "key115"
3) "key113"
4) "key116"
5) "key111"
6) "key118"
7) "key112"
8) "key114"
9) "key117"
127.0.0.1:6379>

This command is very simple to use and just needs a regular expression. But Keys command has no offset and limit mechanism which lead a very big problem.

As we know, Redis is generally known as a singleprocess, single-thread model. If a regular expression hits a lot of keys, Redis will return all the matched data at once which will block all of next commands. It makes keys command should be avoided as much as possible in the production environment.

To address this issue, Redis adds a new command: scan, since version2.8. Compared to keys command, scan has the following features :

  1. Allowing for incremental iteration, returning only a few elements per call with its cursor.
  2. Support to control the returning number of matched elements with limit argument.
  3. It doesn’t block processes.

The scan command is a cursor based iterator, which means that every time a command is called, the cursor returned by the previous call needs to be used as the cursor argument for that call, to continue the previous iteration.

When the cursor parameter of the scan command is set to 0, the server will start a new iteration. When the Redis server returns a cursor with a value of 0 to the user, the iteration is over. This is the only way to determine the end of an iteration, and the end of an iteration cannot be determined by returning a null result set.

Here is an example showing how to use scan in PHP. Before you run the script below, make sure you have installed Redis extension.

<?php
$redis = new Redis();
$redis->connect('localhost');

$it = NULL;
do {
    // Scan for some keys
    $arr_keys = $redis->scan($it);

    // Redis may return empty results, so protect against that
    if ($arr_keys !== FALSE) {
        foreach($arr_keys as $str_key) {
            echo "Here is a key: $str_key\n";
        }
    }
} while ($it > 0);
RSS