0

We want to get seller-contact of some registered seller on our website. We have memcached implemented. So, we first check if cache key exist for this object, we return the cached object, otherwise, we read data from database.

In database, we write data on master database. But we read from the slave database.

Following is API sequence call which creates an issue:

GET API (SellerContact): Its first request so no cache key found. Now, we approach slave database, there's no seller-contact record in the database too, hence, we create a NULL object in memcache as seller-contact doesn't exist, the validity of this key is 30 days.

POST API(SellerContact): Creates new stock entity in the master database, Invalidates the key created with above GET request that has a NULL object.

GET API(SellerContact): If this GET is executed before the master data is synced to the slaved database, this GET will not find any memcache key as above POST request has invalidated it and hence, it will go to the database. Since it's read operation, it will approach Slave database which hasn't got a new object from the master database. So, slave database will get the NULL object and it would get saved in memcache for another 30 days.

Few solution to this problem:

  1. figure out maximum delay b/w master to slave sync process and refresh memcache key after that delay period only i.e. if the maximum possible delay is 5 minutes, then for any memcache key clear request, clear it after 5 minutes of the requested clear datetime.
  2. Once we receive the POST request, we create new Memcache key instantaneously once record is updated in the master database. This will create overhead for POST request and few unwanted keys might be created.

Is there any better solution to this problem?

Sahil Sharma
  • 3,847
  • 6
  • 48
  • 98
  • Usually (2), where you `set` after writing to master and `add` after reading from a slave. Option (3) is to have the cache be a slave of the master, e.g. Kafka + change-data-capture to tail the WAL and push records into the cache. This avoids all races and lets you do similar to other data slave stores (search, analytics, etc) – Ben Manes Apr 18 '18 at 07:33
  • " Kafka + change-data-capture to tail the WAL and push records into the cache." what is this exactly. Could you please elaborate? – Sahil Sharma Apr 18 '18 at 07:45
  • There are many custom implementations of this, e.g. [1](https://www.confluent.io/blog/bottled-water-real-time-integration-of-postgresql-and-kafka/), [2](https://engineeringblog.yelp.com/2016/08/streaming-mysql-tables-in-real-time-to-kafka.html), [3](https://aws.amazon.com/blogs/database/streaming-changes-in-a-database-with-amazon-kinesis/). – Ben Manes Apr 18 '18 at 07:49

0 Answers0