I ran this code last week with Jupyter Notebook and it worked rather quickly. However, this week I've run into issues with it taking forever (more than an hour)to write JSON data to a file. The code works, but I was curious if maybe the way I've written it, is causing it to run slow??
consumer_key = hidden
consumer_secret = hidden
access_token = hidden
access_secret = hidden
auth = tweepy.OAuthHandler(consumer_key, consumer_secret)
auth.set_access_token(access_token, access_secret)
# set Twitter's rate limit
api = tweepy.API(auth, wait_on_rate_limit = True, wait_on_rate_limit_notify = True)
# write the querying JSON data into tweet_json.txt
with open('tweet_json.txt','a',encoding = 'utf8') as f:
for tweet_id in twitter_archive['tweet_id']:
try:
tweet = api.get_status(tweet_id, tweet_mode = 'extended') # set mode to extended
json.dump(tweet._json, f)
f.write('\n')
except:
print('error')