0

When I am comparing 2 json files in different buckets I used etag in order to compare. I loaded the json file into table in dynamodb and now I want to compare that the data is same. Do I have the option to do it? This is what I did for comparing the 2 files between different buckets.

conn = client('s3')  # again assumes boto.cfg setup, assume AWS S3
    for key2 in conn.list_objects(Bucket='freedom-meital')['Contents']:
        for key in conn.list_objects(Bucket='freedom-meital3')['Contents']:
            if key['Key'] == key2['Key']:
                if key['ETag'] != key2['ETag']:
                    print(key['Key']+" is not equal between the 2 buckets")
                else:
                    print(key['Key']+" is equal between the 2 buckets")
mburesh
  • 1,012
  • 5
  • 15
  • 22
Tamar
  • 13
  • 11
  • I would put these in a dict (or two) using Key as key and ETag as the value. That way you can do set operations to find those in both/one. At the moment none of your code uses DynamoDB so I don't understand that aspect of the question. – Andy Hayden Apr 26 '18 at 13:09
  • I know it is not using. I want suggestion how to change it in order to match it to what exist in the dynamodb? Can I get the data from the table and calculate the etag of it? – Tamar Apr 26 '18 at 13:30
  • see [ETag algorithm used by s3](https://stackoverflow.com/a/19896823/1240268) – Andy Hayden Apr 26 '18 at 14:34
  • I was thinking on exporting the data in the table in to s3 by data pipeline and compare the etag of the 2 files. Is that a good idea? – Tamar Apr 26 '18 at 14:55
  • IMO it depends how much data you're talking about, but that could work. – Andy Hayden Apr 26 '18 at 14:57
  • Do i have an option to run existing data pipeline with python in order my whole scrpty will be automatically? – Tamar Apr 26 '18 at 15:12
  • I don't understand the question, best bet is to update this question with some code to do it. – Andy Hayden Apr 26 '18 at 16:11

0 Answers0