S3 giving me NoSuchKey error even when the key exists

Question

This is my boto3 command for getting the object with a specific key from an S3 bucket:

resp = s3client.get_object(Bucket='<>-<>', Key='MzA1MjY1NzkzX2QudHh0')

It gives the following error:

botocore.errorfactory.NoSuchKey: An error occurred (NoSuchKey) when calling the GetObject operation: The specified key does not exist.

I have checked in the bucket, and the key actually exists

Did I miss something or did I do something wrong here?

Answer 1

You have a %0A at the end of your URL; that's a line separator.

Answer 2

Since you know the key that you have is definitely in the name of the file you are looking for, I recommend using a filter to get objects with names with your key as their prefix.

s3 = boto3.resource('s3')
bucket = s3.Bucket('cypher-secondarybucket')
for obj in bucket.objects.filter(Prefix='MzA1MjY1NzkzX2QudHh0'):
    print obj.key

When you run this code, you will get the key names of all the files that start with your key. This will help you find out what your file is exactly called on S3.

Answer 3

A generic answer that may useful to those who are thinking about file paths and may be new to the AWS S3 terminology. Not getting the "name" and "key" right will often lead to an exception with the message An error occurred (NoSuchKey)... as posted in this question.

Let's say that you have a JPEG file stored in some "path" in a bucket. Navigating to this object on the AWS console shows the S3 URI to be:

s3://my-bucket/some/very/long/path/my-image.jpeg

You could read the my-image.jpeg object in Python with this basic example:

import boto3

s3client = boto3.client('s3', region_name='us-east-1')

bucket_name = 'my-bucket'
object_key = 'some/very/long/path/my-image.jpeg'

try:
    s3obj = s3client.get_object(Bucket=bucket_name, Key=object_key)
except Exception as e:
    print(f"Error reading key {object_key} from bucket {bucket_name}: {e}")
else:
   print(f"Got object: {s3obj}")

Answer 4

Another possible issue that I came across that was causing a line separator in my url for an object was that in one of AWS documentation for getting the key of an object shows this code as an example to get the key of an object.

foreach ($results as $result) {
    foreach ($result['Contents'] as $object) {
        echo $object['Key'] . PHP_EOL;
    }
}

The problem is the PHP_EOL at the end. Just remove it and the line separator goes away.

$object['Key'] . PHP_EOL; --> $object['Key'];

Answer 5

from urllib.parse import unquote
key = "MzA1MjY1NzkzX2QudHh0%0A"

key = unquote(key)

This will remove all special characters from any key.