0

I have a scenario where I need to sync my MySQL data into redis cache every 2 minutes.

Basically I have 2 tables categories and articles table. Every article belongs to some particular category. Retrieval : I need to fetch articles of particular section and sometimes limiting it.

I saw the 5 data structures available in Redis but getting confused to choose one of them which will be the appropriate for these requirements.

Every 2 minutes the entire data as to be removed and inserted.

So what is the best way I can go on with it.

Sharath
  • 2,348
  • 8
  • 45
  • 81
  • Can you explain why you need this so we can make an appropriate answer? Your clients only get data from the redis? it's to "sort" the data available? – khanou Nov 24 '15 at 16:01
  • can you explain `fetch articles of particular section and sometimes limiting it` , like for a specific category ? – Rabea Nov 24 '15 at 16:47
  • I have categories and articles. Data is stored in Amazon MySql RDS and every 3 and 15 minutes we parse articles from rss and store in db. So the rds is constantly engaged. There is android, ios, mobile site which requests for the articles so the Rest Api calls are only for fetching through node.js web services. We have around 60 to 70 sections and in each section we planning to put around 100 articles. There is paging concept in mobile site so i need to give only 10 records at a time and for apps its 20 articles per call from the Cache. – Sharath Nov 24 '15 at 16:47
  • Rabee yes its for specific category. Basically they send the section id so all articles in the section should be given 10 at a time a kind of paging concept. But max is 100 each section contains. Bringing all the data to cache will be of no use and size factor. – Sharath Nov 24 '15 at 16:50
  • There is no sort concept so while syncing itself we do an order by timestamp and get the latest articles of each section, so while giving in a response its latest 10,20 records and so on. – Sharath Nov 24 '15 at 16:53

1 Answers1

2

One suggestion that can serve your scenario is to have:

  • Hash tables to contain your articles and categories.

  • Sorted Set to work as an index for the latest articles per category.

From the above we will be:

  • Adding objects to hash tables inside Redis for categories and articles. this will make you benefit from a data structure with a constant time complexity in average, as mentioned here.

    HMSET article_with_id_{1234} field1 value1 .. etc

    HMSET category_with_id_{567} field1 value1 .. etc

  • Now you will need the structure to connect them together, Start creating sorted sets as one category -> many articles, below is an example:

    ZADD category_{category_id} {Sorting Score - it could be the article_id in descending order or the timestamp as you mentioned} {article ID}

Now you should have an index that you can refer to per category. so pulling data will be like :

ZREVRANGE category_{category_id} 0 10

Pulling the sorted set of the highest scoring 10 articles from that category. now this also will help in pagination.

From the article IDs you collected, you can pull the detailed info of the articles from their hash with an HGET

Rabea
  • 1,938
  • 17
  • 26