AWS S3 "Access Denied" on GetObject operation (using AES-256 Server Side Encryption)

Question

I have two AWS accounts and I'm trying to access S3 objects in Account A from Account B. The objects in question were uploaded as a result of Elasticache's copy-snapshot operation, meaning that the root user of Account A is not the true owner. I added the following policies:

The Bucket policy on Account A:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::account_b_id:user/user_x"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::bucket-a-name",
                "arn:aws:s3:::bucket-a-name/*"
            ]
        }
    ]
}

The IAM policy applied to user_x on Account B:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:GetObject",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": [
                "arn:aws:s3:::bucket-a-name",
                "arn:aws:s3:::bucket-a-name/*"
            ]
        }
    ]
}

Here's where some strange things started happening. Making a call similar to this:

aws s3api get-object --bucket bucket-a-name --key backup.rdb localbackup.rdb

I notice the operation ONLY succeeds iff there is no Server Side encryption enabled in the console. By default, every file backed up from Elasticache is encrypted under the S3 AES-256 type, not KMS. Until I disable the encryption, I will always get the error:

An error occurred (AccessDenied) when calling the GetObject operation: Access Denied

However as demonstrated I have given (what I believe to be) sufficient permissions to access these objects. What is going on? How can I access these objects?

I should also note that when I make that very same call from a user with AdminstratorAccess policy on Account A, the operation is successful with no errors.