Trouble authenticating a request to a Google Workflows callback from a Cloud Function

Question

I have a data pipeline set up with Google Workflows where I have a step to wait for a Cloud Function that is triggered from Pub/Sub to complete by using the callback mechanism documented here.

The Workflow and Cloud Function are both deployed with the same service account (which has the Workflows Invoker role) and are in the same project.

I am getting the following HTTP error when I try and call the callback URL: requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://workflowexecutions.googleapis.com/v1/projects

The Workflow is defined like so:

main:
  steps:
    - init:
        assign:
          - project: ${sys.get_env("GOOGLE_CLOUD_PROJECT_ID")}
    - create-callback:
        call: events.create_callback_endpoint
        args:
            http_callback_method: "GET"
        result: callback_load
    - init-callback:
        assign:
          - message: {"name": "check_load_finished", "callback": "${callback_load.url}"}
          - base64Msg: ${base64.encode(json.encode(message))}
    - check-load-finished:
        call: googleapis.pubsub.v1.projects.topics.publish
        args:
          topic: ${"projects/" + project + "/topics/data-scripts"}
          body:
            messages:
              - data: ${base64Msg}
    - await-callback:
        call: events.await_callback
        args:
            callback: ${callback_load}
            timeout: 1200
        result: callback_request

The cloud function calls the following function after it has finished operation:

import google.auth.transport.requests
import google.oauth2.id_token
import requests

def send_callback(callback_url):
    auth_req = google.auth.transport.requests.Request()
    id_token = google.oauth2.id_token.fetch_id_token(auth_req, callback_url)
    print(f"ID Token: {id_token}")

    headers = {
        "accept": "application/json",
        "Authorization": f"Bearer {id_token}",
    }

    r = requests.get(
        url=callback_url,
        headers=headers,
    )
    r.raise_for_status()
    print(f"Callback sent to '{callback_url}'")

I used this reference to try to get the access token from within the Cloud Function.

Any help appreciated!

TL/DR;

I have tried to generate the access token programatically within the Cloud Function as implied here and here
I expected to be able to request the callback URL from within the Cloud Function so that the Workflow will continue
I instead get an HTTPError 401 Client Error: Unauthorized for url

What's the permission of the Coud Function runtime service account? You shouldn't have the role "Workflows Invoker role" on it. That's the reason! — guillaume blaquiere, May 13 '23 at 20:42

score 1 · Answer 1 · edited May 15 '23 at 12:06

After a lot of trial and error, here is the solution to the above if anyone runs into this:

import google.auth
import google.auth.transport.requests
import google.oauth2.credentials

def generate_access_token():
    credentials, project_id = google.auth.default(
        scopes=["https://www.googleapis.com/auth/cloud-platform"]
    )
    request = google.auth.transport.requests.Request()
    credentials.refresh(request)
    access_token = credentials.token
    return access_token

Trouble authenticating a request to a Google Workflows callback from a Cloud Function

1 Answers1