-2

My use case is: I have trained model which i want to use for infer small messages. Not sure about where should i keep my models in cloud run.

  1. inside container
  2. On cloud storage and download it at the time of container start
  3. Mount cloud storage as local directory and use it

I am able to write and run code successfully for option 1 and 2. Tried option 3 but not lucky there. I am using this link https://cloud.google.com/run/docs/tutorials/network-filesystems-fuse Actually here my entry point is an pub sub event. thats where i am not able to make it working.

But before exploring more into it i would like to know about which approach is better here. or is there any other better solution.

Djai
  • 188
  • 10
  • 1
    Stack overflow discourages opinion-based questions because there's no correct answer. – DazWilkin Sep 06 '22 at 00:02
  • 1
    For simplicity and because the model is static, I'd combine it into the image. The model will always be required by the container and so there's lower value in separating it from the code and higher "cost" from it potentially being inaccessible to the container when needed and requiring a separate network pull. If the model changes, you can create a new container image. – DazWilkin Sep 06 '22 at 00:03
  • Your CI|CD pipeline should, of course, trigger a new image build if either the code or the model changes. But, from your customers' perspective, the two aren't separable. – DazWilkin Sep 06 '22 at 00:04
  • 2
    The Cloud Run contract requires your container to initialize and respond to HTTP requests within a specific time (4 minutes). Downloading 2 GB of data is not a good idea for cold start performance or from a cost perspective. Embed the model in your container. Note: your items 2 and 3 are basically the same thing. You are downloading the 2 GB model via different methods. What service are you using to run the model? Some can directly read from Cloud Storage. – John Hanley Sep 06 '22 at 00:24
  • Option 1: faster to start, the model file won't take space in the container memory (only the loaded version. With option 2, you have to download the file (take place in memory) + load it (again take place in memory), consistent versioning (roll back are fast, safe and easy) – guillaume blaquiere Sep 06 '22 at 07:25
  • @JohnHanley What is mean by "What service are you using to run the model" ? could you elaborate your question a bit. – Djai Sep 06 '22 at 09:44
  • Cloud Run is a service. Vendors offer ML services to run models. Which one are you using such as Google Cloud ML? – John Hanley Sep 06 '22 at 17:41
  • As of now we are not using any. We have our own static model for NLP. – Djai Sep 07 '22 at 18:53

1 Answers1

0

Thanks for valuable comments, it helped a lot.

If model is static better to club it with container. downloading it from storage bucket or mounting FS will download model again whenever we spin new container.

Djai
  • 188
  • 10