Limit tweepy stream to a specific number

Question

class listener(StreamListener):

def on_status(self, status):
    try:
        userid = status.user.id_str
        geo = str(status.coordinates)
        if geo != "None":
            print(userid + ',' + geo)
        else:
            print("No coordinates")
        return True
    except BaseException as e:
        print('failed on_status,',str(e))
        time.sleep(5)

def on_error(self, status):
    print(status)


auth = OAuthHandler(ckey, csecret)
auth.set_access_token(atoken, asecret)

twitterStream = Stream(auth, listener())
twitterStream.filter(locations=[-97.54,32.55,-97.03,33.04])

I have this script for my tweepy stream, and it works perfectly. However, it keeps going until I terminate it using 'ctrl+c'. I tried adding a counter to "on_status" but it does not increment:

 class listener(StreamListener):

def on_status(self, status):
    i = 0
    while i < 10:
        userid = status.user.id_str
        geo = str(status.coordinates)
        if geo != "None":
            print(userid + ',' + geo)
            i += 1

No matter where I put the increment, it repeats itself. If I add "i=0" before the class I get an error:

RuntimeError: No active exception to reraise

Any idea how I can make the counter to work with streaming? The Cursor that comes with tweepy does not work with streaming, as far as I know at least.

score 1 · Accepted Answer · edited Jun 15 '15 at 14:10

1

Your while logic is not working properly because Tweepy internally calls the on_status() method whenever it receives data. So you can't control the flow of by introducing a conditional inside an already running infinite loop, The best way is to create a new variable inside the class, which gets instantiated when the listener object is created. And increment that variable inside the on_data() method.

class listener(StreamListener):

    def __init__(self):
        super().__init__()
        self.counter = 0
        self.limit = 10

    def on_status(self, status):
        try:
            userid = status.user.id_str
            geo = str(status.coordinates)
            if geo != "None":
                print(userid + ',' + geo)
            else:
                print("No coordinates")
            self.counter += 1
            if self.counter < self.limit:
                return True
            else:
                twitterStream.disconnect()
        except BaseException as e:
            print('failed on_status,',str(e))
            time.sleep(5)

edited Jun 15 '15 at 14:10

Servy

202,030
26
332
449

answered Jun 09 '15 at 01:36

ZdaR

22,343
7
66
87

Thank you for your response. I still get the same error: `RuntimeError: No active exception to reraise` – Leb Jun 09 '15 at 01:44
have a look @Leb https://github.com/tweepy/tweepy/issues/570 and http://stackoverflow.com/questions/28869442/unraised-exception-using-tweepy-and-mysql – ZdaR Jun 09 '15 at 01:45
Sorry for not mentoning it before, I did change it to `raise exception` as suggested but didn't fix it. – Leb Jun 09 '15 at 01:47
Try using simply `except e` ? @Leb – ZdaR Jun 09 '15 at 01:48
Same issue as previous situations. – Leb Jun 09 '15 at 02:14

Limit tweepy stream to a specific number

1 Answers1

Linked