1

I've come across an interesting (not to say infuriatingly annoying) issue. I'm fairly new to Redis but I know how to operate the basics.

Basically I have a cluster with 3 masters and each master has 2 slaves each across three servers.

I have a bunch of normal keys and I try to clean all keys based on a pattern across all servers so I've run something like this on the master nodes:

for server in "${servers[@]}" do
    ./redis-cli -h ${server} -p ${master_port} KEYS 'pattern:*' | xargs -I {} ./redis-cli -h ${server} -p ${master_port} DEL {}
done

Now this works nicely, it looks like all the keys get deleted as they should however, when I run the KEYS 'pattern:*' command again on each server, keys belonging to master 1 appears in the list of master 2. If I try to access the key with get <key> I properly get a moved notice and if I do the same thing using the -c follow parameter I get correctly redirected and recieve a nil back.

I've tried this same thing with scan using both lua and ruby script but both output the should-be-deleted keys on the node that didn't have the keys in the first place.

Is there any way to delete these keys for good so I can get a trustworthy count/evidence that they're gone?

Almund
  • 5,695
  • 3
  • 31
  • 35

1 Answers1

0

No, that cli loop you executed should return a boolean 1 integer for each key within each server if there is a match to begin with. If you want to be sure all the keys are deleted just simply run KEYS pattern:* for each server to detect any residue remaining while in interactive mode on the terminal.

Also, try using the -c flag when deleting the keys from the masters. I ran the command from my terminal and just to double-check the redirected keys are truly removed, I used the Node.JS wrapper and ran a simple client.keys('pattern', (err, res)=>console.log(res)) and came with an empty array indicating the keys no longer exist. Or you can do a simple flushall command to remove the keys

akiespenc
  • 305
  • 3
  • 8
  • Hi! Thank you very much for your response. The loop I have actually does return a 1 for each remove row, well, or a MOVED unless I apply the `-c` command to make sure it follows the redirect and deletes from the actual master. The problem is that after I've deleted the keys they still appear when I check as you say using the `KEYS pattern:*` on the node that doesn't contain them (different hash slots). – Almund Feb 12 '18 at 11:38