2

I have ~100GB of data running on several Memcached servers, but we're migrating all of our infrastructure to Amazon AWS so I'd like to find a way to replicate this information on these new servers.

The problem I have right now is that to commit all the information to a database I need around 12-14 hours, and while this happens the app. has to go down (we get more data/unit of time than what we can save to the database).

Our main concern is for the app to stay alive during this period so I was wondering if you can suggest any tool that allows us to copy the data from one memcached server to another without having to bring servers down or have to worry about doing it by hand.

Federico B.
  • 131
  • 1
  • 1
  • 4

2 Answers2

5

See this link:

http://horicky.blogspot.com/2009/10/notes-on-memcached.html

Where it says:

When one of the server crashes, all entries owned by that server is lost. Higher resilience can be achieved by storing redundant copies of data in different servers. Memcached has no support for data replication. This has to be taken care by the application (or client lib).

Note that the default server hashing algorithm doesn't handle the growth and shrink of the number of servers very well. When the number of servers changes, the ownership equation (key mod N) will all be wrong. In other words, if the crashed server needs to be taken out from the pool, the total number of servers will be decreased by one and all the existing entries needs to be redistributed to different server. Effectively, the whole cache (among all server) is invalidated even when just one server crashes.

There is "repcached":

http://repcached.lab.klab.org/

but that a patchset for memcached. I should think that would be tricky to implement against your production app.

gm3dmo
  • 10,057
  • 1
  • 42
  • 36
  • Hi, did not understand why adding a server results in cache invalidation; is the hash table in client library create a new entry if a particular entry is not found. – Krishna Kumar May 24 '13 at 17:13
0

I use haproxy + mcrouter + mcseed for high availability.

Haproxy connects the client to an available server. Mcrouter handles replicating writes to multiple machines and Mcseed pre-populates a new node from a surviving one.

enter image description here

symcbean
  • 21,009
  • 1
  • 31
  • 52