2

A system is being implemented using microservices. In order to decrease interactions between microservices implemented "at the same level" in an architecture, some microservices will locally cache copies of tables managed by other services. The assumption is that the locally cached table (a) is frequently accessed in a "read mode" by the microservice, and (b) has relatively static content (i.e., more of a "lookup table" vice a transactional content).

The local caches will maintain synch using inter-service messaging. As the content should be fairly static, this should not be a significant issue/workload. However, on startup of a microservice, there is a possibility that the local cache has gone stale.

I'd like to implement some sort of rolling revision number on the source table, so that microservices with local caches can check this revision number to potentially avoid a re-synch event.

Is there a "best practice" to this approach? Or, a "better alternative", given that each microservice is backed by it's own database (i.e., no shared database)?

SoCal
  • 801
  • 1
  • 10
  • 23

2 Answers2

0

In my opinion you shouldn't be loading the data at start up. It might be bit complicated to maintain version.

Cache-Aside Pattern

Generally in microservices architecture you consider "cache-aside pattern". You don't build the cache at front but on demand. When you get a request you check the cache , if it's not there you update the cache with latest value and return response, from there it's always returned from cache. The benefit is you don't need to load everything at front. Say you have 200 records, while services are only using 50 of them frequently , you are maintaining the extra cache that may not be required.

Let the requests build the cache , it's the one time DB hit . You can set the expiry on cache and incoming request build it again.

If you have data which is totally static (never ever change) then this pattern may not be worth a discussion , but if you have a lookup table that can change even once a week, month, then you should be using this pattern with longer cache expiration time. Maintaining the version could be costly. But really upto you how you may want to implement.

https://learn.microsoft.com/en-us/azure/architecture/patterns/cache-aside

Imran Arshad
  • 3,794
  • 2
  • 22
  • 27
  • I don't see cache-aside as being useful for this, as there is zero guarantee that a locally cached row is synch'd with the source. Also, the local cache is required to satisfy references from other tables, tables local to, and controlled by, the microservice. So, in effect, all the rows in the local cache are required by the microservice on start-up (i.e., the content of the local cache is not driven by requests on the microservice). But, thanks for pointing out the cache-aside pattern. It was an interesting read. – SoCal May 02 '19 at 19:22
0

We ran into this same issue and have temporarily solved it by using a LastUpdated timestamp comparison (same concept as your VersionNumber). Every night (when our application tends to be slow) each service publishes a ServiceXLastUpdated message that includes the most recent timestamp when the data it owns was added/edited. Any other service that subscribes to this data processes the message and if there's a mismatch it requests all rows "touched" since it's last local update so that it can get back in sync.

For us, for now, this is okay as new services don't tend to come online and be in use same day. But, our plan going forward is that any time a service starts up, it can publish a message for each subscribed service indicating it's most recent cache update timestamp. If a "source" service sees the timestamp is not current, it can send updates to re-sync the data. This has the advantage of only sending the needed updates to the specific service(s) that need it even though (at least for us) all services subscribed have access to the messages.

We started with using persistent Queues so if all instances of a Microservice were down, the messages would just build up in it's queue. There are 2 issues with this that led us to build something better:

1) It obviously doesn't solve the "first startup" scenario as there is no queue for messages to build up in

2) If ANYTHING goes wrong either in storing queued messages or processing them, you end up out of sync. If that happens, you still need a proactive mechanism like we have now to bring things back in sync. So, it seemed worth going this route

I wouldn't say our method is a "best practice" and if there is one I'm not aware of it. But, the way we're doing it (including planned future work) has so far proven simple to build, easy to understand and monitor, and robust in that it's extremely rare we get an event caused by out-of-sync local data.

meshtron
  • 1,110
  • 9
  • 19
  • I like the idea of a timestamp-based comparison, as this opens up the potential (as you indicated) to minimize the number of rows required to be used to re-synch tables. I'm currently leaning towards having subscribers ask for current version on startup, but having a provider send a message on startup doesn't seem like a bad idea either (i.e., do both). – SoCal May 05 '19 at 03:23
  • Curious...how are you handling "deletes"? – SoCal May 06 '19 at 18:04
  • @SoCal We use a flag on the record to indicate it's status. So, "delete" is essentially just a modification to the row. EDIT to add: It's up to the receiving service how it handles the delete, no reason it couldn't actually delete the row(s) when this flag changes. But, we don't actually delete them in any current use case. – meshtron May 06 '19 at 20:17