how to properly close a tweepy stream

Question

I'm trying to figure out how to properly close an asynchronous tweepy stream.

The tweepy streaming module can be found here.

I start the stream like this:

stream = Stream(auth, listener)
stream.filter(track=['keyword'], async=True)

When closing the application, I try to close the stream as simple as:

stream.disconnect()

This method seems to work as intended but it seems to have one problem: the stream thread is still in the middle of the loop (waiting/handling tweets) and is not killed until the next loop, so when the stream receives a tweet even after the app has closed, it still tries to call the listener object (this can be seen with a simple print syntax on the listener object). I'm not sure if this is a bad thing or if it can simply be ignored.

I have 2 questions:

Is this the best way to close the stream or should I take a different approach?
Shouldn't the async thread be created as a daemon thread?

Why not use Twython instead? Its the official library for Python from Twitter. — Games Brainiac, Oct 13 '13 at 15:15

score 3 · Answer 1 · answered Dec 30 '15 at 10:02

I had the same problem. I fixed it with restarting the script. Tweepy Stream doesn't stop until the next incoming tweet.

Example:

import sys
import os

python=sys.executable

time.sleep(10)

print "restart"
os.execl(python,python,*sys.argv)

I didn't find another solution.

score 2 · Answer 2 · answered Nov 01 '13 at 02:50

I am not positive that it applies to your situation, but in general you can have applicable entities clean up after themselves by putting them in a with block:

with stream = Stream(auth, listener):
    stream.filter(track=['keyword'], async=True)
    # ...
# Outside the with-block; stream is automatically disposed of.

What "disposed of" actually means, it that the entities __exit__ function is called.
Presumably tweepy will have overridden that to Do The Right Thing.
As @VooDooNOFX suggests, you can check the source to be sure.

score 0 · Answer 3 · answered Oct 31 '13 at 22:36

This is by design. Looking at the source, you will notice that disconnect has no immediate termination option.

def disconnect(self):
    if self.running is False:
        return
    self.running = False

When calling disconnect(), it simply sets self.running = False, which is then checked on the next loop of the _run method

You can ignore this side effect.

score 0 · Answer 4 · answered Feb 27 '17 at 13:49

Instead of restarting the script, as @burkay suggests, I finally deleted the Stream object and started a new one. In my example, someone wants to add a new user to be followed, so I update the track list this way:

stream.disconnect() # that should wait until next tweet, so let's delete it
del stream
# now, create a new object
stream = tweepy.Stream( auth=api.auth, listener=listener )
stream.userstream( track=all_users(), async=True )

how to properly close a tweepy stream

4 Answers4