Unique, safe short codes in redis (using node)

Question

I have users accessing a URL with a short code they're provided, e.g. example.com/ABCD. I'm running node and using node_redis module, although I don't think it matters much. What is the safe way to generate unique codes?

I can generate something randomly and check if it's in use, but given the async nature of my calls to redis, there's a possibility that it would then be used somewhere else before being saved back to redis.

A few thoughts:

I'm not worried about the limited namespace, I can extend it if needed but likely won't
... But the goal of keeping the code short (easy to type on mobile) means I want to keep the namespace small, increasing the likelihood of a collision.
I suppose I could keep the list of used (or unused, for that matter) codes in memory, which isn't the worst thing, but seems like a strange move.
I know I could make redis auto-increment, but I don't want to use numbers (I don't want to be sequential).

If I don't want to keep the list in memory, should I switch data stores? Am I correct in thinking that keeping the list in memory is ok but not great?

score 1 · Accepted Answer · edited May 23 '17 at 11:57

Please see this answer of mine from July 2013: https://stackoverflow.com/a/17597649/1253312

I will copy it here for safe keeping:

I recommend using Redis for this task, actually. It has all the features that make this task suitable for its use. Foremost, it is very good at searching a big list for a value.

We will create two lists, buffered_ids, and used_ids. A cronjob will run every 5 minutes (or whatever interval you like), which will check the length of buffered_ids and keep it above, say, 5000 in length. When you need to use an id, pop it from buffered_ids and add it to used_ids.

Redis has sets, which are unique items in a collection. Think of it as a hash where the keys are unique and all the values are "true".

Your cronjob, in bash:

log(){ local x=$1 n=2 l=-1;if [ "$2" != "" ];then n=$x;x=$2;fi;while((x));do let l+=1 x/=n;done;echo $l; }
scale=`redis-cli SCARD used_ids`
scale=`log 16 $scale`
scale=$[ scale + 6]
while [ `redis-cli SCARD buffered_ids` -lt 5000 ]; do
    uuid=`cat /dev/urandom | tr -cd "[:alnum:]" | head -c ${1:-$scale}`
    if [ `redis-cli SISMEMBER used_ids $uuid` == 1]; then
        continue
    fi
    redis-cli SADD buffered_ids $uuid
done

To grab the next uid for use in your application (in pseudocode because you did not specify a language)

$uid = redis('SPOP buffered_ids');
redis('SADD used_ids ' . $uid);

edit actually there's a race condition there. To safely pop a value, add it to used_ids first, then remove it from buffered_ids.

$uid = redis('SRANDMEMBER buffered_ids');
redis('SADD used_ids ' . $uid);
redis('SREM buffered_ids ' . $uid);

This is really really helpful. I didn't think of using sets to keep track of the codes -- makes perfect sense. Thank you! — jack_was_taken, Apr 29 '14 at 01:28

Unique, safe short codes in redis (using node)

1 Answers1