0

i am receiving bucket name and key as an empty string which causes script to fail. I am trying to retrieve filename which caused the trigger by in eventbridge from S3 but unable to get it.


aws events put-targets --rule s3-to-rds --targets '{
    "Id": "",
    "Arn": "ecs cluster ARN",
    "RoleArn": "eventbridge role ARN with permissions",
    "EcsParameters": {
        "TaskDefinitionArn": "task ARN",
        "TaskCount": 1,
        "LaunchType": "FARGATE",
        "NetworkConfiguration": {
            "awsvpcConfiguration": {
                "Subnets": [
                    ""
                ],
                "AssignPublicIp": "DISABLED"
            }
        },
        "Group": "default"
    },
    "InputTransformer": {
        "InputPathsMap": {
            "s3-bucket": "$.detail.requestParameters.bucketName",
            "s3-key": "$.detail.requestParameters.key"
        },
        "InputTemplate": "{
            \"containerOverrides\": [
                {
                    \"name\": \"containername\",
                    \"environment\": [
                        {
                            \"name\": \"S3_BUCKET_NAME\",
                            \"value\": \"<s3-bucket>\"
                        },
                        {
                            \"name\": \"S3_OBJECT_KEY\",
                            \"value\": \"<s3-key>\"
                        }
                    ]
                }
            ]
        }"
    }
}'

part of python code to retrieve env variables

S3_BUCKET_NAME = os.environ.get('S3_BUCKET_NAME')
logging.info(f"S3_BUCKET_NAME: {S3_BUCKET_NAME}")
S3_OBJECT_KEY = os.environ.get('S3_OBJECT_KEY')
logging.info(f"S3_BUCKET_NAME: {S3_OBJECT_KEY}")

s3 = boto3.client('s3')
response = s3.get_object(Bucket=S3_BUCKET_NAME, Key=S3_OBJECT_KEY)

this is the sample i've used for retrieving enviroment variables and dynamic retrieval.

this is the output in cloudtrail log:

August 30, 2023 at 17:03 (UTC+2:00) response = handler(**kwargs)    tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^   tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/handlers.py", line 284, in validate_bucket_name  tess
August 30, 2023 at 17:03 (UTC+2:00) raise ParamValidationError(report=error_msg)    tess
August 30, 2023 at 17:03 (UTC+2:00) botocore.exceptions.ParamValidationError: Parameter validation failed:  tess
August 30, 2023 at 17:03 (UTC+2:00) Invalid bucket name "<$s3-bucket>": Bucket name must match the regex "^[a-zA-Z0-9.\-_]{1,255}$" or be an ARN matching the regex "^arn:(aws).*:(s3|s3-object-lambda):[a-z\-0-9]*:[0-9]{12}:accesspoint[/:][a-zA-Z0-9\-.]{1,63}$|^arn:(aws).*:s3-outposts:[a-z\-0-9]+:[0-9]{12}:outpost[/:][a-zA-Z0-9\-]{1,63}[/:]accesspoint[/:][a-zA-Z0-9\-]{1,63}$"    tess
August 30, 2023 at 17:03 (UTC+2:00) api_params = self._emit_api_params( tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 1043, in _emit_api_params   tess
August 30, 2023 at 17:03 (UTC+2:00) self.meta.events.emit(  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/hooks.py", line 412, in emit tess
August 30, 2023 at 17:03 (UTC+2:00) return self._emitter.emit(aliased_event_name, **kwargs) tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^    tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/hooks.py", line 256, in emit tess
August 30, 2023 at 17:03 (UTC+2:00) return self._emit(event_name, kwargs)   tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/hooks.py", line 239, in _emit    tess
August 30, 2023 at 17:03 (UTC+2:00) Traceback (most recent call last):  tess
August 30, 2023 at 17:03 (UTC+2:00) File "/app/main.py", line 97, in <module>   tess
August 30, 2023 at 17:03 (UTC+2:00) response = s3.get_object(Bucket=S3_BUCKET_NAME, Key=S3_OBJECT_KEY)  tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 535, in _api_call   tess
August 30, 2023 at 17:03 (UTC+2:00) return self._make_api_call(operation_name, kwargs)  tess
August 30, 2023 at 17:03 (UTC+2:00) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ tess
August 30, 2023 at 17:03 (UTC+2:00) File "/usr/local/lib/python3.11/site-packages/botocore/client.py", line 928, in _make_api_call  tess
August 30, 2023 at 17:03 (UTC+2:00) 2023-08-30 15:03:52,840 - INFO - S3_BUCKET_NAME:    tess
August 30, 2023 at 17:03 (UTC+2:00) 2023-08-30 15:03:52,840 - INFO - S3_BUCKET_NAME:    tess

Fixed it by replacing values of these keys "s3-bucket": "$.detail.requestParameters.bucketName", "s3-key": "$.detail.requestParameters.key" to "s3-bucket": "$.detail.bucket.name", "s3-key": "$.detail.object.key"

Vandalism
  • 23
  • 4
  • Your InputTemplate has `` but your error message says `<$s3-bucket>`. Do you have stray `$` characters in your actual InputTemplate? – Mark B Aug 30 '23 at 17:13
  • @MarkB i've tried with and without the $, the output with $ returns $s3-bucket, without it returns empty string – Vandalism Aug 30 '23 at 21:03
  • I think the one returning empty string is correct. That indicates that you have this part wrong: $.detail.requestParameters.bucketName"` – Mark B Aug 31 '23 at 09:33

0 Answers0