1

Is there any definition of the retention policy behaviour for static assets during deployments?

In other words if I'm running v1 and deploying v2, are v1's assets assured to be available for the entirety of an app deploy command? I'm particularly interested where multiple services are defined.

I've looked at these articles and they don't provide much insight into the behaviour;

https://cloud.google.com/appengine/docs/standard/python/getting-started/serving-static-files

https://cloud.google.com/appengine/docs/standard/python/how-instances-are-managed

https://cloud.google.com/appengine/docs/standard/python/getting-started/hosting-a-static-website

To be more specific I have a project with 3 services;

  • assets.yaml - service: assets, provides static images, css, js, etc and acts as cookie-less domain.
  • site.yaml - service: default, provides static html for "brochure site".
  • app.yaml - service app, go app provides /healthz, /readyz, and /a/*.
  • dispatch.yaml - for route specification.
# dispatch.yaml

# prod assets
- url: "a.example.net/*"
  service: assets

# local development assets
- url: "*/css/*"
  service: assets

# local development assets
- url: "*/js/*"
  service: assets

# local development assets
- url: "*/media/*"
  service: assets

- url: "*/a/*"
   service: app

- url: "*/healthz"
  service: app

- url: "*/readyz"
  service: app

Given v1 has the asset css/bootstrap-4b2.min.css and v2 has css/bootstrap-4b3.min.css.

When I deploy v2 and v1 is the current running version with the command below;

gcloud app deploy --project=$PROJECT dispatch.yaml assets.yaml default.yaml app.yaml --quiet --version $VERSION

What behaviour can I expect from App Engine?

  1. Both static files will be available for the duration of the project deployment.
  2. Both static files will be available for the duration of the service deployment.
  3. Both static files will be available indefinitely.
  4. Something else/it depends?
Nathan
  • 553
  • 4
  • 11
  • Should we assume no assets have been cached at the edge location we're fetching from? – gravitation Jan 18 '18 at 11:27
  • What is the content of your `dispatch.yaml` file? The answer depends on it. – Dan Cornilescu Jan 18 '18 at 18:28
  • @BrandonThomson I'm not sure depending on Cache-Control headers is ideal because it's a count down. If I have a Cache-control of 300s but I deploy at 299s it could result in someone making a request at 299s receiving a v1's HTML and 404 for CSS. – Nathan Jan 18 '18 at 20:33
  • I'll add an example. @DanCornilescu can you clarify how that would affect the behaviour of how files are added/removed from FrontEnd? I thought it was purely responsible for routing where the static_files/upload did the actual file upload to either Google Frontend or AppEngine. I think my answer is to put the assets and Google Cloud Storage or reference the "version" in host portion of the asset URL for now. – Nathan Jan 18 '18 at 20:39
  • (but I was hoping for something all inclusive) – Nathan Jan 18 '18 at 20:52

1 Answers1

2

To start with, there is no deployment at the app level, each service is deployed independently from each-other. You just emulate an app-level deployment by deploying all the services (almost) at the same time, in the same CLI invocation, but technically they're still deployed independently.

The availability of the static files is dictated by the lifetime of the service version publishing them. So as long as both v1 and v2 versions of the asset service are deployed, both css/bootstrap-4b2.min.css and css/bootstrap-4b3.min.css will be potentially available. But with a twist: it depends on which version of the service receives the request - only the service version serving a particular version of the file should/will serve that file version. Otherwise the behaviour of the service would not be deterministic - you'd always get a 404 on the local server if that version doesn't serve the file, but on GAE you'd get either a 404 or the file, depending if the version serving that file is still deployed, which would be a bug.

Deciding which version of a service receives the request depends on the URL routing rules (see How Requests are Routed - this is where the dispatch.yaml content comes into play) and on the traffic distribution configuration for that service (see Migrating Traffic and Splitting Traffic).

For example, let's assume that asset version v1 is configured to receive all the traffic for the service. In such case:

  • a request to https://your_app.appspot.com/css/bootstrap-4b2.min.css should work, v1 gets the request via dispatch.yaml rule
  • a request to https://your_app.appspot.com/css/bootstrap-4b3.min.css should return 404, v1 gets the request via dispatch.yaml rule
  • a request to https://v1-dot-assets-dot-your_app.appspot.com/css/bootstrap-4b2.min.css should work, v1 gets the request via soft routing
  • a request to https://v1-dot-assets-dot-your_app.appspot.com/css/bootstrap-4b3.min.css should return 404, v1 gets the request via soft routing
  • a request to https://v2-dot-assets-dot-your_app.appspot.com/css/bootstrap-4b2.min.css should return 404, v2 gets the request via soft routing
  • a request to https://v2-dot-assets-dot-your_app.appspot.com/css/bootstrap-4b3.min.css should work, v2 gets the request via soft routing

If you configure traffic splitting between versions, the results in the 1st 2 cases listed above can be reversed, depending in which percentile of the split the request falls into. But that should normally not be a problem, since usually the static assets are referenced from dynamic pages and GAE attempts to keep flows within the same percentile.

Finally, all the above doesn't take into account caching issues (at various levels). But those only affect you if you modify file contents while keeping the same filenames (i.e.URLs), which is not the case in this question.

Dan Cornilescu
  • 39,470
  • 12
  • 57
  • 97
  • Thanks that pretty much answers my question. I wouldn't expect caching to be an issue because the "root" of the request is what drives the asset requests and the assets will have globally unique names. I'm ok with stale reads (e.g. v1 w/ bootstrap-4b2.css) what I want to avoid is a 404 where v1's required assets are no longer there. The safest approach seems like it would be to either keep all versions of assets in the deployment. – Nathan Jan 19 '18 at 21:30
  • "To start with, there is no deployment at the app level, each service is deployed independently from each-other. You just emulate an app-level deployment by deploying all the services (almost) at the same time, in the same CLI invocation, but technically they're still deployed independently." I figured this to be the case but wasn't sure if there was something that made it an "atomic" operation. – Nathan Jan 19 '18 at 21:32