Session created, but not used by boto client

Question

I have the following code, which allows me to list information for all resources in all of my accounts in ~/.aws/config:

#!/bin/bash

#Output file
OUTFILE="out.txt"

###############################################################################################
#EXECUTION SECTION - Shouldn't have to change anything below this line.
###############################################################################################

rm -f $OUTFILE

for AWS_PROFILE in `grep '\[profile ' ~/.aws/config | awk {'print $2'} | sed 's/\]//g'`; do
    echo "====================================================================" >> $OUTFILE
    echo "!!==> $AWS_PROFILE " | tee -a $OUTFILE
echo "!!==> $AWS_PROFILE " | tee -a $OUTFILE
    for region in `aws --profile $AWS_PROFILE ec2 describe-regions --all-regions --query 'Regions[].RegionName' --output text`
    do
        echo "region = ${region}" >> $OUTFILE
        aws --profile $AWS_PROFILE resourcegroupstaggingapi get-resources --region ${region} --query 'ResourceTagMappingList[].ResourceARN' >> $OUTFILE
    done
done

Again, this works; it prompts me for the MFA token when it queries each account, and I at least have the raw output with no errors.

I need to port this to python, for expandability, in line with the rest of our code base. I start off with the following:

#!env python3.9

from pprint import pprint

import boto3
import boto, boto3
from boto.sts import STSConnection
from botocore.exceptions import ClientError

role_arn =  'arn:aws:iam::account-number-removed:role/role-name-here'

# Prompt for MFA time-based one-time password (TOTP)
mfa_TOTP = input("Enter the MFA code: ")

sts_connection = STSConnection()

tempCredentials = sts_connection.assume_role(
    role_arn=role_arn,
    role_session_name="AssumeRoleSession1",
    mfa_serial_number="arn:aws:iam::account-number-of-bastion-account-here::mfa/my-email-here",
    mfa_token=mfa_TOTP
)

assumed_role_session = boto3.Session(
    aws_access_key_id=tempCredentials.credentials.access_key,
    aws_secret_access_key=tempCredentials.credentials.secret_key,
    aws_session_token=tempCredentials.credentials.session_token
)

print(assumed_role_session.client("sts").get_caller_identity())
client = boto3.client('resourcegroupstaggingapi', )

regions = assumed_role_session.get_available_regions('ec2')

for region in regions:
    print(region)
    try:
        client = boto3.client('resourcegroupstaggingapi', region_name=region)
        pprint([x.get('ResourceARN') for x in client.get_resources().get('ResourceTagMappingList')])
    except ClientError as e:
        print(f'Could not connect to region with error: {e}')
    print()

When I run it, the script starts off and gets the session as expected. Sensitive information removed.

➜  aws git:(master) ✗ ./list-resources.py
Enter the MFA code: removed
{'UserId': 'alphanumeric-characters-here:AssumeRoleSession1', 'Account': 'account-number-removed', 'Arn': 'arn:aws:sts::account-number-removed:assumed-role/role-name-here/AssumeRoleSession1', 'ResponseMetadata': {'RequestId': 'id-here', 'HTTPStatusCode': 200, 'HTTPHeaders': {'x-amzn-requestid': 'id-here', 'content-type': 'text/xml', 'content-length': '451', 'date': 'Thu, 06 Jan 2022 19:48:11 GMT'}, 'RetryAttempts': 0}}

However, when it goes through the regions and tries to list resources, it gets errors, seemingly because it's still using the token from the initial account, not the token from the session of the assumed role. Also, the resources it does list are for the bastion account, not the account of the assumed role.

af-south-1
Could not connect to region with error: An error occurred (UnrecognizedClientException) when calling the GetResources operation: The security token included in the request is invalid.

ap-east-1
Could not connect to region with error: An error occurred (UnrecognizedClientException) when calling the GetResources operation: The security token included in the request is invalid.

ap-northeast-1
[]

...
us-east-1
['arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblah-development_users-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblahUsers-sk-data-index-WriteCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:Users-WriteCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblahUsers-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblahUsers-sk-gsiSk-index-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblahUsers-sk-data-index-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblahUsers-sk-gsiSk-index-WriteCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblah_development_clinton_test-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblah-development_users-WriteCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblah_development_clinton_test-WriteCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblah-development_users-sk-index-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:Users-ReadCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblah-development_users-sk-index-WriteCapacityUnitsLimit-BasicAlarm',
 'arn:aws:cloudwatch:us-east-1:account-number-of-bastion-account-here:alarm:blahblahUsers-WriteCapacityUnitsLimit-BasicAlarm']
....
us-west-2
['arn:aws:networkmanager::account-number-of-bastion-account-here:global-network/global-network-alphanumeric-values-removed']

Note that I read How to fetch all aws resources in all regions in lambda function, with boto3 lib, and I don't think I need to aggregate data here; in fact, probably want to leave them in their regions.

It seems the boto3 client used to query for resources isn't getting the same credentials (token, etc.) as the one for the assumed role. Any thoughts on how I can do this?

score 1 · Accepted Answer · answered Jan 06 '22 at 21:57

The 'resourcegroupstaggingapi' client needs to be created on the assume_role_session, not the default boto3 session:

client = assumed_role_session.client('resourcegroupstaggingapi', region_name=region)

(Note that you have a duplicate client = boto3.client('resourcegroupstaggingapi', ... and should probably remove the first one)

Session created, but not used by boto client

1 Answers1