Twitter api not returning 'created_at' info for tweets

Question

When running the following code using the Python Tweepy library:

query = '$MSFT'
start_time = '2019-01-01T00:00:00Z'
end_time = '2019-02-01T00:00:00Z'
max_results = 10

results = client.search_all_tweets(query=query, max_results=max_results, start_time=start_time, end_time=end_time)

I receive a bunch of tweets. However, these tweets only contain the following data:

tweet.edit_history_tweet_ids
tweet.id
tweet.text

Information like 'created_at' or 'geo' are missing. Anyone have any idea what is going wrong here?

You have to specify the [`tweet_fields`](https://docs.tweepy.org/en/stable/expansions_and_fields.html#tweet-fields-parameter) argument to get more data. Look [here](https://developer.twitter.com/en/docs/twitter-api/data-dictionary/object-model/tweet) to see what fields are available. Right now you're just retrieving the default fields. — Timus, Jan 15 '23 at 20:07
To get the geo information you should specify the [`place_fields`](https://docs.tweepy.org/en/stable/expansions_and_fields.html#place-fields) argument (e.g. `place_fields=['geo']`) and at the same time specify the [`expansions`](https://docs.tweepy.org/en/stable/expansions_and_fields.html#expansions) argument (`expansions='geo.place_id'`). — Timus, Jan 16 '23 at 15:27

Bench Vue · Accepted Answer · 2023-01-16T01:42:18.790

Adding tweet_fields parameter with specific field can see created_at but geo usually missed in tweet due to not added by tweeter.

It can't see most case even if passing geo

I can't test it due to no paid account but I can test recent 7 days result. as attached code and results. The search_recent_tweets() and search_all_tweets() almost same parameters.

Here is documentation.

It is matched V2 low level API in here

GET /2/tweets/search/all

import tweepy

bearer_token ="***** your bearer_token *****"

query = '\$MSFT'
start_time = '2019-01-01T00:00:00Z'
end_time = '2019-02-01T00:00:00Z'
max_results = 10

results = client.search_all_tweets(query=query, 
    tweet_fields=['created_at','geo'],
    max_results=max_results,
    start_time=start_time,
    end_time=end_time)

for result in results.data:
    print(result.data)

import tweepy

bearer_token ="***** your bearer_token *****"

client = tweepy.Client(bearer_token=bearer_token)

query="\$MSFT"
results = client.search_recent_tweets(query=query, 
            tweet_fields=['created_at','geo'],
            start_time='2023-01-12T00:03:00Z',
            max_results=10
            )

for result in results.data:
    print(result.data)

Result

>python search.py
{'created_at': '2023-01-16T01:27:49.000Z', 'text': '@excel_ranger $MSFT', 'edit_history_tweet_ids': ['1614796530795298817'], 'id': '1614796530795298817'}
{'created_at': '2023-01-16T01:27:26.000Z', 'text': 'Live day trading, detailed analysis on stocks with entry, exit, stop losses, price targets,mentoring and live news\n\nhttps://xxx/LzwYLosqbb\n\n$BIDU $SAVE $WORK $T $AAPL $C $MSFT $SPY $FB $CHWY $PTON $DIS $F $ADBE $CSCO $MGM $XOM $AMD $ZI $Z $BA $DOW $NET $PROP $OSTK  $QCOM $ADM https://xxx/n3T6xDyNNj', 'edit_history_tweet_ids': ['1614796435139993607'], 'id': '1614796435139993607'}
{'created_at': '2023-01-16T01:27:10.000Z', 'text': '$SPY $QQQ $DIS $TSLA $SHOP\n$AMD $AAPL $SQ $AMZN \n$EA $SEDG $MA $V $KO $PYPL $RCL $GOOG $NKLA $DKNG $HD $ROKU $NFLX $FB  $GILD $VXX $MSFT\n\nThanks to the discord group for the traders https://xxx/LzwYLosqbb https://xxx/V3pz7tESOk', 'edit_history_tweet_ids': ['1614796367511035904'], 'id': '1614796367511035904'}
{'created_at': '2023-01-16T01:26:56.000Z', 'text': 'RT @DarthDividend23: Great post by @thedividendclub on IG! \n\n$WMT $APD $JNJ $PG $WBA $GILD $O $MRK $UPS $AAPL $COST $SBUX $TGT $LOW $MSFT $…', 'edit_history_tweet_ids': ['1614796311076769794'], 'id': '1614796311076769794'}
{'created_at': '2023-01-16T01:26:52.000Z', 'text': 'RT @TrendSpider: Do the Markets need a Breather? \n\nPrepare for the week ahead \n\nTickers Covered:\n$SPY 00:00\n$QQQ 01:58\n$IWM 04:57\n$BTC 07…', 'edit_history_tweet_ids': ['1614796291812327428'], 'id': '1614796291812327428'}
{'created_at': '2023-01-16T01:26:46.000Z', 'text': 'Alerted an entry for $METX win of 121%. \nhttps://xxx/LzwYLosqbb\n✅Check us out\n\n\n\n$NAKD $TSLA $AAPL $NIO $SPY $INTC $GE $SNDL $NXTD $KGC $ZOM\n$FCEL\n$FEYE \n$ZM\n$AAL \n$PLTR \n$CCL \n$MSFT \n$PFE\n$PLUG \n$WFC \n$AMD \n$VXX \n$CNSP $QQQ https://xxx/OZFbKDpaHp', 'edit_history_tweet_ids': ['1614796270048010243'], 'id': '1614796270048010243'}
{'created_at': '2023-01-16T01:25:50.000Z', 'text': "STOCK, OPTIONS updates, alerts Free chatroom\nDon't forget to take a trial! \nhttps://xxx/LzwYLosqbb\n\n$SPY $BABA $DVAX $ACB $OSTK $TRIL $LK $CODX $SAVE $GSX $INO $KSS $PENN $NVAX $NIO $AAL $NKLA $MSFT $AAPL $AMZN $TSLA $CCL $BILI $CVNA $DAL $TWTR https://xxx/cT6Lc9Yhpv", 'edit_history_tweet_ids': ['1614796034487603200'], 'id': '1614796034487603200'}
{'created_at': '2023-01-16T01:25:36.000Z', 'text': '$MSFT at $220 feels like stealing candy from a baby.', 'edit_history_tweet_ids': ['1614795972751785985'], 'id': '1614795972751785985'}
{'created_at': '2023-01-16T01:25:16.000Z', 'text': '$DXY Index looks to test 101.60 as odds of less-hawkish Fed bets soar https://xxx/9PYMyxzvpt  $CRM $DXY $NIO $BABA $BTC.X $ETH.X $BLK $COIN $BNB.X $AAPL $TSLA $MULN $CEI $SPY $DJIA $QQQ $WMT $MSFT $PFE $MRNA $AZN $ABNB $AMD $BNTX $BA $COP $PDD $COST $GM $META $AMZN $NFLX', 'edit_history_tweet_ids': ['1614795889251422208'], 'id': '1614795889251422208'}
{'created_at': '2023-01-16T01:24:52.000Z', 'text': '$AAPL Trade idea\nhttps://xxx/LzwYLosqbb\n\n$AMC $SPY $GME $QQQ $MU $MSFT $AMD $PTON $AMZN $CRM $XLF $XLE $TSLA $AAL https://xxx/rDouPd3hNj', 'edit_history_tweet_ids': ['1614795790412648450'], 'id': '1614795790412648450'}

Thank you very much. Was not aware that the solution was so simple, but was stuck on this for many hours! — Tes Taccount, Jan 16 '23 at 17:23
No problem, please don't forget vote me. It will encourage me to keep sharing. — Bench Vue, Jan 16 '23 at 17:58
I did upvote, but it told me I need more reputation.. Will return to this post when I hit 15 reputation :) — Tes Taccount, Jan 17 '23 at 12:17
No, just press up arrow in my answer, will give me a point (15) even if you less 15 points. Then it will turn to green color. — Bench Vue, Jan 17 '23 at 12:27
I think I pressed the right button now :) Btw, I made another post regarding the Twitter API just now. It regards the possibility over random sampling using the search_all_tweets method. Would u perhaps also know the answer to this? The post: https://stackoverflow.com/questions/75146457/is-there-a-way-to-retreive-a-random-sample-of-tweets-using-the-full-archive-sear — Tes Taccount, Jan 17 '23 at 12:35
Thanks, You did correctly, now it turn to green color checked mark. All tweet question, you can do paging token. I am using free account so I can't test it but you can figure out from my other answer. Can you try it? https://stackoverflow.com/questions/74986120/how-does-the-api-get-retweeter-ids-actually-work-tweepy-python/75045630#75045630 — Bench Vue, Jan 17 '23 at 12:42

Twitter api not returning 'created_at' info for tweets

1 Answers1