Python tweepy streamer

Question

I'm running the tweepy streamer using the on_data function, and I cannot get the full tweet even is the mode is set to extended: when I run my def on_data passing the text argument only, then I get the truncated tweets, and that's fine, but when I am passing full_text, then I get the following error: KeyError: 'full_text'. I think I a doing what I understood from the Tweepy API docs, but I am still getting errors. Can anyone help me here?

    def on_data(self, data):
        json_load = json.loads(data)
        tweet = json_load["full_text"]
        user = json_load["user"]["screen_name"]
        print(user, tweet)
        try:
            with open("data_file.txt", 'a') as df:
                df.write(tweet + "\n")
            return True
        except BaseException as e:
            print("Error on_data %s" % str(e))
        return True


# letting the user decide the subject
subj = input("please enter the hashtag to follow separated by commas: ")
stream_listener = MyStreamListener()
tweets = tweepy.Stream(auth = api.auth, listener=stream_listener, tweet_mode="extended")
tweets.filter(track=[subj], is_async=True)```

score 2 · Answer 1 · answered Jul 02 '20 at 10:20

2

In the standard streaming API, extended Tweets are represented automatically, and there is no need to pass tweet_mode="extended" (in fact this option is invalid and will be ignored on the streaming endpoint - it only works on the regular REST endpoints that return Tweets).

You should find that the full_text value sits inside another section of the Tweet data called extended_tweet. Check out this answer.

You could try your existing code with e.g.

    tweet = json_load["extended_tweet"]["full_text"]

or, as suggested in the linked answer:

def on_status(self, status):
    try:
        text = status.extended_tweet["full_text"]
    except AttributeError:
        text = status.text

answered Jul 02 '20 at 10:20

Andy Piper

11,422
2
26
49

Hi @andy-piper , thanks for this. I tried to the `tweet = json_load["extended_tweet"]["full_text"]` before and I am still getting the `KeyError` and I do not understand why. – mrb Jul 02 '20 at 10:35
Also, when trying to use `on_status` as you suggested, I get another error: `tweet_mode="extended"` – mrb Jul 02 '20 at 10:44
I wondering if I am putting the `tweet_mode="extended"` in the right place? Now I put it in the definition of my stream, but seems is not read? – mrb Jul 02 '20 at 10:57
1

I have deleted the `on_data` function, and I am progressing with the `on_status` : something like this: `text = status.extended_tweet["full_text"]` which seems to work now, I got also confused cause all the retweets [RT] are still capped at 140 characters, so I think I am good now, thanks. – mrb Jul 02 '20 at 12:51

Python tweepy streamer

1 Answers1