How to deal with container restarts when pushing data to Open Policy Agent

Question

At the company I work for, we're considering whether to use Open Policy Agent (OPA) for fine grained authZ.

For loading data to OPA there are a number of approaches listed on the website under External Data. For the scenarios we're dealing with, I'd like to use a combination of JWT Tokens and Push Data (depending on the specific use case). As a side note; also thinking of initially hosting OPA as a standalone service instead of sidecar containers (even though were using K8s and Istio), as this seems a bit simpler (conceptually) and will be easier to get buy in from broader IT. Depending on how things go, we could migrate to the sidecar approach later on.

WRT the Push Data approach;

1. If data is stored in memory (when using Push Data approach); how would we deal with the scenario when OPA crashes or gets redeployed?
2. Assuming multiple OPA replicas; how do we ensure data gets propagated to all OPA containers?

Answer 1

If data is stored in memory (when using Push Data approach); how would we deal with the scenario when OPA crashes or gets redeployed?

Similar to OPA's bundle feature, and the integration with its /health API for ready-checks, you would likely want to build a similar feature into your tooling that is pushing data into OPA.

The idea being that the tool you build (the "replicator" as described on https://www.openpolicyagent.org/docs/latest/external-data/#option-4-push-data) could know if it has synchronized things with OPA, and you would potentially gate declaring that instance of OPA as "ready" until that point.

Assuming multiple OPA replicas; how do we ensure data gets propagated to all OPA containers?

Typically there is a certain amount of eventual consistency in these systems. The strategy taken by OPA with bundles is that each instance of OPA will be able to report whether it is "ready" (mentioned above), but then maybe more importantly it also has a set of features to integrate with the decision logger and provenance feature so that you can audit what version of the bundle was used for any policy evaluation. The idea being that even if some OPA instance is out of sync (which is bound to happen in a large distributed system) you can keep track of it. Similar to the answer above you would likely want to build something similar for your "replicator".

All that being said there is a trend, most of this stuff is largely solved if you can use bundles. If you are still in the planning phase make sure to take a look at whether or not you need to push data directly to OPA.