What's a good technique to store a time-dependent metric in Redis?

Question

I have some metrics (like counts of logged in users, or SQL queries, or whatever), and I want to gather some time-dependent stats on a regular basis.

For example I want to know how many users were registered in some particular year, month, week, day or even hour.

I thought maybe Redis can be a good fit. But I can't imagine a good strategy for storing such stats. The only idea I have is to store independent counters for days, weeks, etc, and bump them all at once.

How do you do it? I need a good trick. Or maybe Redis isn't any good for my task.

Answer 1

If all you need is a count for each period, the multiple counter approach you suggest is a good one. Incrementing 5 counters in a single pipelined transaction is O(1), while set operations are O(log n + m) with potentially large values of n/m.

The set solution Frank suggested does have its place - I use something similar where I need to know which actions happened rather than just how many. Obviously storing details of each action takes more memory than the counters, but with the amount of RAM typically available these days you can store millions of records before that becomes a problem.

Answer 2

I would just use a sorted sets where the score is the timestamp in seconds since the epoch (unix time). Say you have a sorted set of logins and you want to see how many logins occured in the year 2010, just convert 20101231 23:59:59 and 20100101 00:00:00 to seconds and use those are the max and min arguments to zcount.

The obviously difficulty here is handling the time conversion yourself, but its actually very easy because it the standard Unix format. You can use the date command with %S (on linux at least) or use the system calls time(), localtime() and mktime(), as well as any of the myriad ways available within specific languages that are built on top of these system calls.

I am sure there is some equivalent paradigm in Windows, but that I don't have any experience there.