1

The boto3 s3 get_object function documentation (which is autogenerated from AWS service definition files) describes the IfMatch argument as

Return the object only if its entity tag (ETag) is the same as the one specified, otherwise return a 412 (precondition failed).

However when I try to use the IfMatch argument I'm unable to get S3 to respond either with a non-200 HTTP status code or by throwing an exception.

This example code

import boto3
import hashlib
import json

BUCKET = 'your-bucket-name-goes-here'
data = {'foo': 'bar', 'baz': 'buz'}
payload = json.dumps(data, sort_keys=True, indent=4).encode('UTF-8')
etag = hashlib.md5(payload).hexdigest()
print("Locally computed etag : {}".format(etag))
client = boto3.client('s3')
response = client.put_object(
    Body=payload,
    Bucket=BUCKET,
    ContentType='application/json',
    Key='/test.json')
etag_with_quotes = response['ETag']
print("Etag returned from put_object: {}".format(etag_with_quotes))

response = client.get_object(
  Bucket=BUCKET,
  Key='/test.json',
  IfMatch=etag
)
print("Etag returned from get_object: {}".format(response['ETag']))
print("HTTP status code : {}".format(response['ResponseMetadata']['HTTPStatusCode']))

response = client.get_object(
  Bucket=BUCKET,
  Key='/test.json',
  IfMatch=etag_with_quotes
)
print("Etag returned from get_object: {}".format(response['ETag']))
print("HTTP status code : {}".format(response['ResponseMetadata']['HTTPStatusCode']))

produces this output

Locally computed etag : d14e36174d57c305bd9d7c171e669ac8
Etag returned from put_object: "d14e36174d57c305bd9d7c171e669ac8"
Etag returned from get_object: "d14e36174d57c305bd9d7c171e669ac8"
HTTP status code : 200
Etag returned from get_object: "d14e36174d57c305bd9d7c171e669ac8"
HTTP status code : 200

How is one supposed to use the IfMatch argument to avoid downloading a file if the local copy is the same?

gene_wood
  • 1,960
  • 4
  • 26
  • 39

1 Answers1

3

IfMatch downloads the object if the ETag matches. To download only if different, use IfNoneMatch.

Note also that this only works for objects not uploaded using multipart uploads. Multpart uploads use a different ETag algorithm containing a hex md5 hash of the binary (not hex) md5 hashes of the individual parts, then a -, and then the number of parts. You can successfully calculate these yourself, but it's more complicated.

Michael - sqlbot
  • 169,571
  • 25
  • 353
  • 427
  • Excellent. Confirmed that this works. Switching to use `IfNoneMatch` results in a `ClientError: An error occurred (304) when calling the GetObject operation: Not Modified` exception which I can catch and know that the file hasn't changed. Thanks! – gene_wood Jul 29 '19 at 15:05