Redis SCARD returning the wrong results?

Question

I'm adding a large number of data points into a redis set:

$t = 0;
$redis = Redis::Connection();
foreach($adv as $a) {
    $t = $t + 1;
    print($t); //Prints to log

    $test = $redis -> sadd('database/ab/AL', $a -> id);
    print($test); //Prints to log
}

When I call redis->scard('database/ab/AL') I get the result 9832, The answer I should be getting is 9866

$t is a counter I put in to check how many iterations the loop is doing, and $t is 9866 after running the loop, which is weird considering scard is returning 9832

Then I thought maybe there are duplicates being added, so I logged the response from sadd

1 [2015-06-29 16:24:55] local.INFO: 1 [] []
2 [2015-06-29 16:24:55] local.INFO: 1 [] []
3 [2015-06-29 16:24:55] local.INFO: 1 [] []
4 [2015-06-29 16:24:55] local.INFO: 1 [] []
5 [2015-06-29 16:24:55] local.INFO: 1 [] []
6 [2015-06-29 16:24:55] local.INFO: 1 [] []
...
9861 [2015-06-29 16:24:59] local.INFO: 1 [] []
9862 [2015-06-29 16:24:59] local.INFO: 1 [] []
9863 [2015-06-29 16:24:59] local.INFO: 1 [] []
9864 [2015-06-29 16:24:59] local.INFO: 1 [] []
9865 [2015-06-29 16:24:59] local.INFO: 1 [] []
9866 [2015-06-29 16:24:59] local.INFO: 1 [] []

There are no zeros in the entire log, which means that each element being added is unique. There are also 9866 log calls which contradicts the scard result. I have tried checking with redis-cli and I still get the wrong results. What gives?

Did you log the `SADD` responses with pipelining commented out too? I'm almost positive that you have duplicates and that the count is right, but pipelining can mess with the responses in this case — Itamar Haber, Jun 29 '15 at 16:47
Yes, these results were taken **without** pipelining. I removed the commented sections to avoid confusion. I'm certain that there are no duplicates(though I could be wrong). The data I'm iterating over, `$adv`, is from an SQL query with the `DISTINCT` modifier — RN_, Jun 29 '15 at 16:52
Suggestion: before you run your code, run `MONITOR` in a different connection to your database. This will show you all the commands rushing in... capture that and it should be fairly simply to see where things went wrong from it. — Itamar Haber, Jun 29 '15 at 17:05
Is this behaviour repeatable? Are you tried to drop set and fill it again? Get you same wrong result or other wrong result? — vp_arth, Jun 29 '15 at 17:07
Yup, I have tried it over and over again with the same `9832` result. So far this is the only table that returns this. There are no duplicates in the table, but when I run the raw SQL query, there are indeed `9866` rows. — RN_, Jun 29 '15 at 17:12
@ItamarHaber I ran `MONITOR` as you suggeste and there are `9866` entries, yet `SCARD` returns the wrong result — RN_, Jun 29 '15 at 17:49
@ItamarHaber turns out it was because some keys were lowercase than uppercase! — RN_, Jun 29 '15 at 18:08
LOL - at least we got a good measure of excitement :) Upvoting your answer! — Itamar Haber, Jun 29 '15 at 18:09
@vp_arth I definitely should have, but I modified the code I posted for simplicity's sake. Next time I won't do that. I have two mistakes to learn from now :) — RN_, Jun 29 '15 at 18:15
I'm just little angry :) I remove my downvotes. Yet another proof, that there are not miracles. — vp_arth, Jun 29 '15 at 18:16
@vp_arth exactly... and sometimes it takes more than one pair of eyes and a single troubled soul to get to the right answer :) — Itamar Haber, Jun 29 '15 at 18:36

score 2 · Answer 1 · answered Jun 29 '15 at 18:06

2

I was actually keying the values with a variable:

$redis->sadd('database/ab/state:'.$a->state, a->id);

turns out some states were in lowercase, which sent those values to another key.

Fix: $redis->sadd('database/ab/state:'.strtoupper($a->state), a->id);

now I get the correct number, 9866, when I call SCARD

Always double check your key names!

answered Jun 29 '15 at 18:06

RN_

878
2
11
30

I broke my redis with huge amount of tests, but you just lied in the question. `Always provide actual code in the SO questions` – vp_arth Jun 29 '15 at 18:13

Redis SCARD returning the wrong results?

1 Answers1