I want to set the limit of maximum number of Tweets

Question

I am very new to python. I'm using tweepy library to scrape tweets via twitter streaming API. but it seems like connection gets broken after running for an hour. I want to know if there is any way to stop the program from running before the connections get broken. In short limiting the tweets.

I have tried the .items method but it did'nt work as it gives the name Error.

  from tweepy import Stream
  from tweepy import OAuthHandler
  from tweepy.streaming import StreamListener


  ckey="xxxxxxxxxxxxxxxxxxxxxxxxxxx"
  csecret="xxxxxxxxxxxxxxxxxxxxxx"
  atoken="xxxxxxxxxxxxxxxxxxxxx"
  asecret="xxxxxxxxxxxxxxxxxxxxxxxxxxx"

  class listener(StreamListener):

    def on_data(self, data):
        print(data)
        return(True)

    def on_error(self, status):
       print status

  auth = OAuthHandler(ckey, csecret)
  auth.set_access_token(atoken, asecret)

  twitterStream = Stream(auth, listener())
  twitterStream.filter(track=["Obama"])

thanks

thanks for replying "'Connection broken: IncompleteRead(0 bytes read)', IncompleteRead(0 bytes read)" — target021, Aug 18 '19 at 16:38

Kashan · Accepted Answer · 2019-08-19T22:31:03.773

To solve your connection issue take help from this:

Tweepy Connection broken: IncompleteRead - best way to handle exception? or, can threading help avoid?

To achieve tweets limitation you can return False from the class def on_data method, when the desired number of tweets are fetched. Set max number of tweets in the init method and use try and except for error handling. This might help

def __init__(self):
    super().__init__()
    self.max_tweets = 10
    self.tweet_count = 0

def on_data(self, data):
    try:
     data
    except TypeError:
        print(completed)
    else:
     self.tweet_count+=1
     if(self.tweet_count==self.max_tweets):
       print("completed")
       return(False)
     else:
      decoded = json.loads(data)

I want to set the limit of maximum number of Tweets

1 Answers1