0

My website currently employs a system wherein the description for a particular post comes from memcache. If the memcache fails the system fetches the same data from the database. I wish to remove the database from the picture completely. For this I was looking at high availability and fault tolerant data stores and RIAK seemed to fit the bill. I want to have RIAK as a failover for memcache.

However the site has a huge amount of traffic and hence I wanted to know how many RIAK servers will I need. I have a daily traffic of around 150 million page views.

Thanks In Advance!!!

  • 1
    As this depends a lot on the size distribution of your data, the amount of data you plan to store as well as the specification of the hardware you intend to deploy Riak on, I would recommend setting up a cluster, applying all the recommended tuning steps and then benchmarking it. – Christian Dahlqvist Nov 22 '13 at 08:04

2 Answers2

1

Given the information you provide, it is really hard to give you a concrete answer here. The above assumption of 1736 req/sec is too rough in my opinion, since that traffic could be 10x higher in peak time. Therefore it would be very interesting to know the number of requests per second in that time.

Since you also use memcached as a cache in front of Riak, the next question would be how your cache hit ratio looks like - how many of your requests will be served from memcached, how many will hit your database?

Then also, it will depend a lot on the hardware used, if you're expecting high read/write load on riak, then SSDs would be a must.

Having some experience with riak under high load (significally higher than yours) I can tell you that not the number of requests matters that much (it is amazing how much riak can handle), but more the amount of data you have in riak and the size of the objects you store. The bigger your objects the slower Riak will be. If your objects are definately above 1MB, then please do not consider riak - it's not going to work out. When using Riak, it's definately better to have fewer but faster servers.

If you want to go for riak, I would definately suggest you to do some heavy load testing with jmeter or apache ab to see if riak can handle your traffic. (and even then - maybe slowly fade traffic to your riak installation) that way you can be 100% sure, that nothing is going to burst in production.

Finally, please be aware, that no software is perfect. Switching to Riak is replacing one beast by another one. however, mySQL is a known beast already. there are many people out there who can help you if something goes wrong in your MySQL cluster. If something is wrong with your Riak cluster, you might find little help on the net - and commercial Riak support is very expensive.

Matthias
  • 2,622
  • 1
  • 18
  • 29
0

[EnvelopeMath] 150,000,000 / 24 / 3600 = 1736 Requests/second [/EnvelopeMath]

This is certainly within the range of what Riak can do, but a strong answer still depends on the object size, read/write %, how many objects you will be storing, uniform load distribution vs. peak times, etc.

I would follow @ChristianDahlqvist 's recommendation and setup a 5-node bitcask-backed Riak cluster, and see if it will fit the bill. If you have any questions, please feel free to reach out to the Riak Users Mailing list or the #riak IRC channel on freenode.

Alex Moore
  • 3,415
  • 1
  • 23
  • 39