MLOps - How to refresh a ML model

Question

The image below show the two pipelines that we have in my company to manage the life cycle of a model.

• The first pipeline, "Application", relates to the creation of the application component that host the model and has the inference logic.
• The second, that of "Model", is a pipeline that leads to the generation of the model in binary format.

Together (model and application) will be deployed in our orchestrator (a kubernetes cluster).

I am in a situation where the application logic does not change but the models do. I could find myself in the situation below.

I suppose there are two approach to manage the runtime model refresh on the orchestrator (hope someone suggests me other possibilities that I haven't thought about):

1. In the application logic; The code manages the refresh through a thread by taking the new model.
   • Pros: A new container is not generated
   • Cons: Ability to introduce a bug.
2. Through the pipeline; The pipeline must be triggered by an event (in my case a merge on a git branch) and bring the container by performing a rolling update. The new container at run will load the new model.
   • Pros: Existing process
   • Cons: Each new version of the model should provide for a new build of the container, even if the application logic has not changed.

[Question] Are there any best practices for these cases (perhaps through a system of tags on the images) that someone can suggest me?

Thank you Kipliko

Answer 1

The most seamless way is to do a rolling update via k8s/kubectl. This will require a new container, however, this is considered a best practice, as each container stays atomic and reproducible. Updating the model via threads would be difficult to debug.

Another scenario you could do is blue-green deployment using Istio, and slowly move traffic between the old and new model, although this would require a bit more overhead.