I am trying to output the batch transformer output to another account s3 bucket(batch_output_path is the destination path) however the KMS encryption key is picked from the source account instead of using the default encryption key which is already set in the (test_bucket). The files are getting placed in the destination bucket but I get below error when I try to download the file.
AccessDeniedThe ciphertext refers to a customer master key that does not exist, does not exist in this region, or you are not allowed to access.
How can I enforce S3 bucket to use default encryption kms key of the destination bucket. Below code is from AWS Sagemaker pipeline.py script
batch_output_path= f"s3://test_bucket/folder"
from sagemaker.transformer import Transformer
transformer = Transformer(
model_name=step_create_model.properties.ModelName,
instance_type="ml.c5.xlarge",
instance_count=1,
strategy='MultiRecord',
assemble_with='Line',
max_payload=1,
max_concurrent_transforms=10,
output_path=batch_output_path,
sagemaker_session=pipeline_session
)