Counter grouped by category, author and date in Redis

Question

I am implementing a system that store a large amount of data in a relational DB.

Data can be classified into categories and have an author.

I want to get the number of items grouped by date, category and author and the sum of all items of each category grouped by date.

The system must be near real-time.

E.g. (3 categories, 3 authors, 2 dates)

item1 category1 author1 2015-04-23
item2 category1 author2 2015-04-23
item3 category2 author1 2015-04-23
item4 category1 author1 2015-04-23
item5 category2 author2 2015-04-23
item6 category2 author2 2015-04-24
item7 category3 author1 2015-04-24
item8 category2 author3 2015-04-24
item9 category2 author2 2015-04-24

Results:

2015-04-23:
    category1 author1: 2
    category1 author2: 1
    category1 author3: 0
    category2 author1: 1
    category2 author2: 1
    category2 author3: 0
    category3 author1: 0
    category3 author2: 0
    category3 author3: 0
2015-04-24:
    category1 author1: 0
    category1 author2: 0
    category1 author3: 0
    category2 author1: 0
    category2 author2: 2
    category2 author3: 1
    category3 author1: 1
    category3 author2: 0
    category3 author3: 0

There are about 50 categories and about 50 authors.

How could model this behavior in redis?

Answer 1

Use a Hash per date, use the category and author as field names and keep the counter as value.

For example, for the first item do:

HINCRBY 20150423 1:1 1
            ^    ^ ^ ^
      date -+    | | +- increment (static)
    category id -+ +- author id

Note: I've deliberately used shorter identifiers to save RAM.

To get the data for each date, simply HSCAN the relevant key (be careful with HGETALL because it may take too much time/RAM depending on the Hash's size). To get all date keys you get either use SCAN (never use KEYS) or keep an index of the dates in another data structure (i.e.g. a Set).