Upload GCS File to Google Drive (Airflow)

Question

I am using airflow.providers.google.suite.transfers.gcs_to_gdrive.GCSToGoogleDriveOperator to upload a file from GCS to Google Drive.

Getting 403 Forbidden with reason "insufficientPermissions"

This is the logging. Not sure where and what the issue is. any help is highly appreciated !!!

{gcs_to_gdrive.py:151} INFO - Executing copy of gs://adwords_data/conversion_data.csv to gdrive://google_adwords/
{gcs.py:328} INFO - File downloaded to /tmp/tmpzmk4n2z9
{http.py:126} WARNING - Encountered 403 Forbidden with reason "insufficientPermissions"

Code

    from airflow.providers.google.suite.transfers.gcs_to_gdrive import GCSToGoogleDriveOperator  
    copy_google_adwords_from_gcs_to_google_drive = GCSToGoogleDriveOperator(
        task_id="copy_google_adwords_from_gcs_to_google_drive",
        source_bucket="{}".format(gcs_to_gdrive_bucket),
        source_object="conversion_data.csv",
        destination_object="adwords_data/",
        gcp_conn_id='google_cloud_default',
       dag=dag
    )

In the gcp_conn_id = google_cloud_default. I have added the scope = https://www.googleapis.com/auth/drive

Please edit your question and include your code. Your app has not been authorized with scopes needed to grant you permission to upload fix the scopes — Linda Lawton - DaImTo, Jun 13 '22 at 12:43

score 1 · Answer 1 · answered Jun 16 '22 at 15:03

In your airflow connection, in the scopes field, try adding:

https://www.googleapis.com/auth/drive, https://www.googleapis.com/auth/cloud-platform

You'll also need to ensure the service account has the correct roles assigned (as well as the drive being shared w/ the Service Account's email)

Some of the roles you can add to the service account by going to IAM > click edit on the service account, and add the roles

Upload GCS File to Google Drive (Airflow)

1 Answers1