2

we are starting with our first cloud run project. in the past we used postgres in combination with spring boot. and we had our migrations running via flyway (similar to liquibase) when the app started.

now with cloud run, this approach will maybe hit it's limit because of the following (corner) cases:

  • multiple incoming requests (http, messages) routed to parallel instances that could do the same migration in parallel when bootstrapping the container. that would result in exceptions and retries of failed messages or http errors
  • flyway check on bootstrap would slow down the cold start times additionally every time a container gets started, which could be a lot if we do not have constant traffic with "warm" instances

what would be a good approach having springboot/flyway and postgres as backing database shared accross the instances? a similar problem arises when you replace postgres with a nosql datastore i guess if you want/need to migrate new structures....

right now i can think of:

  • do a migration of the postgres schema as part of the deployment pipeline before the cloud revision gets replaced which would introduce new challenges (rollbacks etc.)

please share your ideas? looking forward to your answers? marcel

niesfisch
  • 63
  • 6
  • I don't have better Idea than yours, and remove the startup check to speed up the instance start. Be careful, Spring Boot not optimized takes long seconds to be up and ready to serve request. I [wrote an article](https://medium.com/google-cloud/3-solutions-to-mitigate-the-cold-starts-on-cloud-run-8c60f0ae7894) to mitigate Cloud Run cold starts – guillaume blaquiere Nov 20 '20 at 13:43
  • @guillaumeblaquiere thanks for your answer. i will have a look at your article. – niesfisch Nov 23 '20 at 08:02

1 Answers1

5

For migrations that introduce breaking changes either on commit or on rollback, it's mandatory to have a full stop, and of course, rollback planned accordingly.

Also pay attention, that a commit/push should not trigger immediately the new migrations. Often these are not part of the regular CI/CD pipeline that goes to production.

After you deploy a service, you can create a new revision and assign a tag that allows you to access the revision at a specific URL without serving traffic.

A common use case for this, is to run and control the first visit to this container. You can then use that tag to gradually migrate traffic to the tagged revision, and to rollback a tagged revision.

To deploy a new revision of an existing service to production:

gcloud beta run deploy myservice --image IMAGE_URL  --no-traffic --tag TAG_NAME

The tag allows you to directly test(or run via this the migration - the very first request) the new revision at a specific URL, without serving traffic. The URL starts with the tag name you provided: for example if you used the tag name green on the service myservice, you would test the tagged revision at the URL https://green---myservice-abcdef.a.run.app

Pentium10
  • 204,586
  • 122
  • 423
  • 502
  • thanks for you answer. i will have a look into your solution. – niesfisch Nov 23 '20 at 08:05
  • looking at your solution that would mean having a copy of the datastore, do the migration and then move the traffic to the new revision (aka green/blue) deployment? if it would be the same instance then you could only apply non breaking changes ... right? – niesfisch Nov 23 '20 at 08:44
  • @niesfisch yes that sounds good, your operational stuff Is unknown for us, so discuss with your team – Pentium10 Nov 23 '20 at 10:17
  • thanks ... already started discussing the stuff :) thanks again. – niesfisch Nov 24 '20 at 07:24
  • While I agree with the notion that DB migrations should be separate from code-deployments, but in case of staging/dev setups it might be a good option to run migrations as part of CI/CD pipeline. In the context of GCP, Cloud Run and Cloud SQL the configuration/support is not documented in a way that is easy for developers. I have added instructions to use ENTRYPOINT in the Dockerfile in this thread (hope it helps in executing if/when needed, depending on your use-case): https://stackoverflow.com/a/69088911/867451 – jdecode Sep 07 '21 at 13:51