0

I'm writing a program that downloads tweets from twitter and stores the information in DynamoDB. It uses the requests python library, API Gateway and AWS Lambda to communicate with the database.

For about three quarters of the tweets, I have no problem with the POST request to the database however, about a quarter of the tweets get an error. Can someone explain the error?

I've included the section of code that deals with the request. The error says an Attribute cannot contain an empty string so in an attempt to deal with that, I've set default values for each field.

        h = {
            'content-type': 'application/json'
        }
        payload = {
            "created_at": d.get('created_at',"a"),# String
            "text": d.get("text","a"),# String
            "id": d.get('id_str',"0"), # String
            "payload": d, #json object
            "coordinates":d.get("geo","a"), # true or json object
            "quality": getQuality(d) # a number
        }
        r = requests.post(url, data=json.dumps(payload), headers=h)

enter image description here

Siunami
  • 417
  • 2
  • 6
  • 13
  • Are you sure the get method sets the default value. Can you print the payload and show whether all the attributes are populated? – notionquest Aug 25 '17 at 09:14
  • created_at - Fri Aug 25 08:54:28 +0000 2017 text - RT @ShadyAHSCult: DM: I feel like Lily is wayyy overdue of being the lead in a season. She's so talented and she honestly deserves it… id - 901004838527488000 coordinates - None quality - 33.3166931993 – Siunami Aug 25 '17 at 09:18
  • This is an example of the values returned. for coordinates, will the value None cause a problem? – Siunami Aug 25 '17 at 09:20
  • No, where is the payload in the print. can you just comment the payload attribute and try to insert the data. – notionquest Aug 25 '17 at 09:22
  • Ok I printed and confirmed that the attributes are populated. Inserting data does not work though. Status code 200 on the request – Siunami Aug 25 '17 at 09:31
  • So, now you are not getting the error message? But you couldn't see the data in DynamoDB table? – notionquest Aug 25 '17 at 10:04

0 Answers0