5

When writing a Cloud Run service, we develop a container that listens on the PORT environment variable to handle an incoming HTTP request. An instance of the container springs to life and handles the request and then lives a while longer after concluding the original request in case there are further requests to arrive. If there are no further requests, GCP shuts down the container. It is in this area that I have a question.

Is there a hook, signal or other indication that the container is being shut down within the container?

In my example, my container wants to cleanly end. Perhaps it wants to close connections or perform some quick flush of cache.

Kolban
  • 13,794
  • 3
  • 38
  • 60
  • I'd try listening for the `SIGTERM` signal and see if it fires. That's part of the standard docker container shutdown sequence and I'd guess something similar would occur here. – Travis Webb Dec 26 '19 at 16:56
  • I found a really good FAQ on Cloud Run which has an answer ... see ... https://github.com/ahmetb/cloud-run-faq#what-is-the-termination-signal-for-cloud-run-services – Kolban Dec 26 '19 at 17:21
  • 1
    Remember that your are billed only when the instance is processing a request, and here, you would like to perform some processing outside a request (not billed). **You need to have a strong use case for justifying this!** In the spirit of Cloud Run, the service is stateless. I can understand that connexions must be closed for preventing any daemon connections to external services, but flashing cache isn't a valid use case in the Cloud Run container contract. – guillaume blaquiere Dec 27 '19 at 13:56
  • @guillaumeblaquiere - That's a great thought!! Thank you for that. It has got me thinking of the corollary ... and that is "Are we billed for the processing/startup time that is incurred prior to receiving the request?". My guess would be a solid yes and opens up a whole new avenue of consideration (for the future). – Kolban Dec 28 '19 at 00:35
  • @guillaumeblaquiere If I capture the sigterm Signal and during the shutdown period if I ping my cloud run service then could it prevent cloud run from shutting down or simply could it stop cold starts? – john mich May 27 '21 at 20:00
  • 1
    @johnmich, If you ping Cloud Run, you can prevent cold start (if it was the latest active instance of your service). I described that in [this article](https://medium.com/google-cloud/3-solutions-to-mitigate-the-cold-starts-on-cloud-run-8c60f0ae7894). But the instance that trigger the shutdown, will shutdown. – guillaume blaquiere May 27 '21 at 20:04
  • @guillaumeblaquiere This is amazing, I will read this article. Thank you for your amazing contribution :D – john mich May 27 '21 at 20:08

2 Answers2

5

An FAQ of great collected Q&A on Cloud Run has been found here. Within that FAQ there is an item which reads:

What is the termination signal for Cloud Run services?

Currently, Cloud Run terminates containers while scaling to zero with unix signal 9 (SIGKILL). SIGKILL is not trappable (capturable) by applications. Therefore, your applications should be okay to be killed abruptly.

A related and important entry also reads:

When will my service scale to zero?

Cloud Run does not provide any guarantees on how long it will keep a service "warm". It depends on factors like capacity and Google’s implementation details.

Some users see their services staying warm up to an hour, or longer.

Opinion

I find it interesting that the story appears to be an immediate SIGKILL. If we take Docker as a basis for a container environment, we can read about docker stop which appears to be the way to cleanly stop a container. In its own description it says:

The main process inside the container will receive SIGTERM, and after a grace period, SIGKILL.

This would appear to indicate that for a normal Docker container stop, the process running the container will receive a signal.

Kolban
  • 13,794
  • 3
  • 38
  • 60
2

SIGTERM (unix signal 15) is now sent to the Cloud Run service and you are given 10 seconds to handle closing any resources etc, if you handle the signal in your code.

See here.

Container instances can be shut down at any time. When a container instance needs to be shut down, new incoming requests are routed to other instances and requests currently being processed are given time to complete. The container instance then receives a SIGTERM signal indicating the start of a 10 second period before being shut down (with a SIGKILL signal). During this period, the container instance is allocated CPU and billed. If the container instance does not catch the SIGTERM signal, it is immediately shut down.

enter image description here

(image of code taken from here)

Graham Polley
  • 14,393
  • 4
  • 44
  • 80