1

I want to execute the following in a bash script:

num_total_keys=0x1FFFF

for ((i = 0; i <= $num_total_keys; i++))
do
    redis-cli SET some_prefix:$i True > /dev/null
done

When I execute this, however, it takes a really long time. Is there a more efficient way to do this?

hermit.crab
  • 852
  • 1
  • 8
  • 20

1 Answers1

2

Rule of thumb:

If your command accept pipelined data/instructions; do not run your command repeatedly in a shell loop, but build all of it before piping to your command as a single call like this:

#!/usr/bin/env sh

i=131071

while [ $i -ge 0 ]; do
  printf 'SET some_prefix:%d True\n' $i
  i=$((i - 1))
done | redis-cli --pipe

Alternatively using Bash's brace expansion:

printf 'SET some_prefix:%d True\n' {131071..0} | redis-cli --pipe
Léa Gris
  • 17,497
  • 4
  • 32
  • 41
  • 1
    These commands run around 10x faster if you use `redis-cli --pipe` Have a look here... https://redis.io/topics/mass-insert – Mark Setchell Oct 05 '21 at 10:40
  • Thank you so much for these tips. I would just like to add that in Linux systems, `\r\n` might need to be used instead of `\n` in the `printf` command. See https://stackoverflow.com/a/35093371/5762785 – hermit.crab Oct 06 '21 at 02:10