How to shutdown a Virtual Machine in GCP after successful execution of code?

Question

I have following

• A repository which contains a code(python) to be executed
• A VM at Google Cloud (debian/ubuntu)
• A scheduler + cloud function to start VM every hour

What I require

• I want to run the code present in repository which will be pulled every hour and runs 15-25 mins and finally after its execution the machine shuts down.

What I already tried:

Using Crontab in machine.

1. used @reboot run_my_script && shutdown -h now
2. Once this is implemented I am not able to access VM anymore because at every start the vm runs the script and shut itself down.

This method not works Example what if my code runs to error or my system requires new dependency

Syncing Cloud scheduler and Crontab

1. used 5 * * * * run_my_script && shutdown -h now
2. Cloud scheduler is set to start my vm at every hour exactly at hour:00:00
3. Crontab starts running the code at hour:05:00 and shutdown after completion (5 minutes to start vm)

This method is running fine But I require a more robust solution(Industry accepted way)

Code Snippets

VM

Name

test_vm

Crontab

5 * * * * sh /home/danish_bansal/workflow.sh

workflow.sh:

#!/bin/sh
sudo rm -r repoName/ || true
sudo git clone https://<token>@github.com/Repo/repoName.git
cd repoName
/usr/bin/python3 Script.py
sudo shutdown -h now

Cloud Scheduler

At 0 * * * * calls cloud function

Cloud Function

from googleapiclient import discovery

def startInstance(r):
    service = discovery.build('compute', 'v1')
    print('VM Instance starting')

    # Project ID for this request.
    project = 'project-name' 

    # The name of the zone for this request.
    zone = 'us-central1-a'  

    # Name of the instance resource to start.
    instance = 'test-vm'

    request = service.instances().start(project=project, zone=zone, instance=instance)
    response = request.execute()

    print(response,'VM Instance started')

Answer 1

You can use Compute Engine Client Library to call instance_client.stop().

This is a much better procedure than using the old Google API Client Library, as this is constantly updated by Google developers.

Here is an example code:

import sys
import typing

import google.cloud.compute_v1 as compute_v1

def stop_instance(project_id, zone, machine_name):
    instance_client = compute_v1.InstancesClient()
    operation_client = compute_v1.ZoneOperationsClient()

    print(f"Stopping {machine_name} from {zone}...")
    operation = instance_client.stop(
        project=project_id, zone=zone, instance=machine_name
    )
    while operation.status != compute_v1.Operation.Status.DONE:
        operation = operation_client.wait(
            operation=operation.name, zone=zone, project=project_id
        )
    if operation.error:
        print("Error during stop:", operation.error, file=sys.stderr)
    if operation.warnings:
        print("Warning during stop:", operation.warnings, file=sys.stderr)
    print(f"Instance {machine_name} stopped.")
    return

stop_instance('project-ID', 'zone', 'machine-name')'

Here is also the official example from Github.

https://github.com/googleapis/python-compute/blob/main/samples/snippets/sample_start_stop.py

Answer 2

I tried to use a different approach to obtain a similar result as you were asking, using Cloud Build Trigger that will get the code directly from the Github repository and Cloud Scheduler to run it at the frequency I prefer.

I begin creating the Trigger following the official documentation “Building repositories from GitHub”:

As is said in the documentation before creating the trigger you must:

• Allow the Cloud Build API
• Have your source ready in your Github repository.
• and be aware to have admin-level permissions in your repository.

Once that you have the prerequisites we can go for the first step, connect the Cloud Build from your GCP project to your Github repository.

You’ll be able to do it by going to the Trigger section from your Cloud Build and clicking on “connect repository”, it will open a window where you’ll have to choose between your repositories and connect them. The official documentation also explains how to sign and authorize the Google Cloud Build App to connect to Google Cloud if you haven’t done it before.

To create the GitHub trigger this time in the Trigger section, instead of choosing “connect repository” you have to select “create trigger”, in this part you will be able to create and set the trigger to be able to run the code from your Github repository. The following steps are the ones that worked for me:

1. Add Name, Description and Tags (as you wish).

2. In the Event part I selected “Manual invocation”.

3. In ”Source” I selected my Github repository from the drop-down and in the ”Revision” part I selected “Branch” (for me it was set automatically once I selected the repository).

4. For the ”Configuration” for “Type” I used “Cloud Build configuration file (yaml or json)” and for the “Location” I used the “Inline” option where in the button that appears as an editor I added the next code:

            Steps:
            - id: "run"
              name: python:3-alpine
              entrypoint: python3
              args:
               - hello.py

With this code what I am setting is the language I need to run my code (Python) and the name of the file where is my code that I will want to run, in this case, hello.py.

5. Then I clicked on “create”.

Once it is created, to test it, you can go again to the Trigger section and in the list you’ll be able to find the trigger that you just have created, you can click the “RUN” button and to check if it works you can go to the Cloud Build History.

In this section you will find a list with all the builds created, choosing the one related to our trigger and opening it you’ll be able to see that the code that was named in the code present in your repository, in my case hello.py, was successfully run.

Why using Cloud Build?

The reason, a part of being able to get the code directly from the Github repository, is because of how Cloud Build works itself, it allows you to stop worrying about having to stop the code, because once the code is executed it will stop by itself.

Now that we demonstrate that we can run our code directly from the Github repository creating a Cloud Build Trigger, we can move on to set it to be able to run it at every hour.

For this we can use Schedule triggers, you’ll be able to set it in the same Trigger section, selecting the trigger that you just have created and going to the three dots menu that is next to the “RUN” button, there you'll find “Run on schedule” and once you select it, it will open a window where you’ll have to enable the “Cloud Scheduler API”, select a “Service Account” and enter the “Frequency” that you will need.

With all these settings I was able to get my code from my repository to run every hour.