Google Service Account credentials stored in AWS SecretManage

Question

in my Aws EC2 instance, I've a python script that interacts with Google Bigquery to perform several operations.

For security reasons, I don't want to use the service account private key as a file in my project, so, I stored it in my AWS Secrets Manager.

In python, it's easy to get the key using boto3:

import boto3
from botocore.exceptions import ClientError
import google.oauth2
    def get_secret():       
       secret_name = "put_key_name_here"
       region_name = "put_region_name_here"
       session = boto3.session.Session()
       client = session.client(service_name='secretsmanager', region_name=region_name)
       get_secret_value_response = client.get_secret_value(SecretId = secret_name)
       a = json.loads(get_secret_value_response['SecretString'])
       return a['credentials'] #HERE I HAVE MY CREDENTIALS STRING
   

now I need to use one of the methods of the Google client. I can't use from_service_account_file but only from_service_account_info https://googleapis.dev/python/google-auth/1.7.0/user-guide.html

but the code goes on error:

dict_secrets={}
dict_secrets["client_email"]="my_account@myproject.iam.gserviceaccount.com"
dict_secrets["token_uri"]="https://oauth2.googleapis.com/token"
dict_secrets["private_key"] = get_secret() #myfunction to get the private_key
my_project='my_google_bigquery_project'
credential_bq = google.oauth2.service_account.Credentials.from_service_account_info(dict_secrets, scopes=["https://www.googleapis.com/auth/cloud-platform"])
client_bq = bigquery.Client(credentials=credential_bq, project=my_project)

ValueError: No key could be detected.

without dict_secrets["private_key"] I received a differ error:

 ValueError: The private_key field was not found in the service account info

Where I am wrong? Exist another way to do these operations? thanks!

Answer 1

From the docs

The response syntax is:

{
    'ARN': 'string',
    'Name': 'string',
    'VersionId': 'string',
    'SecretBinary': b'bytes',
    'SecretString': 'string',
    'VersionStages': [
        'string',
    ],
    'CreatedDate': datetime(2015, 1, 1)
}

So you need so pull out the 'SecretString' from the response -

get_secret_value_response = client.get_secret_value(SecretId = secret_name)
secret_value = get_secret_value_response["SecretString"]

Also, from the Google Service Account docs, the downloaded key has the following format, where private-key is the private portion of the public/private key pair:

{
  "type": "service_account",
  "project_id": "project-id",
  "private_key_id": "key-id",
  "private_key": "-----BEGIN PRIVATE KEY-----\nprivate-key\n-----END PRIVATE KEY-----\n",
  "client_email": "service-account-email",
  "client_id": "client-id",
  "auth_uri": "https://accounts.google.com/o/oauth2/auth",
  "token_uri": "https://accounts.google.com/o/oauth2/token",
  "auth_provider_x509_cert_url": "https://www.googleapis.com/oauth2/v1/certs",
  "client_x509_cert_url": "https://www.googleapis.com/robot/v1/metadata/x509/service-account-email"
}

: