I am using Python and TwythonStreamer to stream statuses from Twitter. Currently, I setup a automatic disconnect by counting the number of collected tweets as below:
First I set this globally
max_tweets = 8000
Then in def on_success(self, tweet) , I have a code snippet:
if (count >= max_tweets):
self.disconnect()
return False
How can I disconnect the stream on exact date and time, let's say, on 'October 12, 2014 00:00:00'
Since the stream process acts like a serverForever server (unless you pass a retry count)
twython/streaming/api.py._request
if self.retry_count and (self.retry_count - retry_counter) > 0:
time.sleep(self.retry_in)
retry_counter += 1
_send(retry_counter)
The only way to stop the process is, do a self.shutdown itself in on_success callback when some condition will be true
OR,
the hard way: wrap the stream with a process, keep its pid or reference ,and kill it from another process (like is commented here)
EDIT: An approach of streamer wrapper and its caller
class Your_Streamer(TwythonStreamer, Process):
def __init__(self, ):
Process.__init__(self)
consumer_key, consumer_secret, oauth_token,oauth_secret = '','','','' #your twitter keys
super(SocialVane_Streamer, self).__init__(consumer_key, consumer_secret, oauth_token,oauth_secret)
def run (self):
keyword_list =[] #the keywords you want to monitorize
if keyword_list:
logger.info("start_monitoring, tracking keywords = %s" % ','.join(keyword_list))
self.statuses.filter(track=','.join(keyword_list))
def on_error(self, status_code, data):
print status_code
# Want to stop trying to get data because of the error?
# Uncomment the next line!
# self.disconnect()
def on_success(self, data):
if 'text' in data:
pass #do what you need
And now from another thread, can be a main method or what you want
stream = Your_Streamer()
if stream.ready():
stream.start()
logger.debug("pid %i created" % stream.pid)
while not your_stop_condition:
import time
time.sleep(1000) #some sleep
stream.terminate()
