1

I am using python-twitter with the Twitter Streaming API to try and collect tweets from the UK which are about a particular topic. The loop I'm using is:

while True:
    try:
        for item in api.GetStreamFilter(locations = UK_BB, stall_warnings = True):
            tweet = parse_tweet(item)
            if relevant(tweet):
                save_tweet(tweet)
    except Exception as err:
        logging.exception("Error at %s:", datetime.datetime.now().strftime("%A, %d. %B %Y %I:%M%p"))

This process works fine for about five minutes, saving relevant tweets. However after about five minutes, every time, data stops coming from the API. I don't receive any exceptions or any unusual messages from the stream. I don't have more than one connection open to the stream. The API credentials seem to be fine and there's nothing wrong with my connection. Restarting the connection will make it work for another five minutes, but then the problem will kick in again.

Does anyone have any idea what might be going wrong here?

Cian Murphy
  • 21
  • 1
  • I haven't used python-twitter, but do you really need to run the streaming command inside the `while True` loop? I had used `Tweepy` and we don't need to call the streaming method inside an infinite loop, it handles internally. – ZdaR Jun 01 '15 at 15:28
  • Interesting point - I guess I did that to handle any errors with connecting to the stream (not that it has!). Just tried it the way you've suggested and am getting exactly the same problem - works for a few minutes and then nothing. – Cian Murphy Jun 01 '15 at 15:36
  • Then you may use Tweepy or ask your query on https://github.com/bear/python-twitter/issues – ZdaR Jun 01 '15 at 15:57
  • Thanks. Put together a tweepy version and have run into exactly the same problem, so I guess this isn't really a python-twitter problem, but something with either my network, my credentials or the streaming API more generally! – Cian Murphy Jun 01 '15 at 16:25
  • Then I guess there is a network issue, if not then try creating a new twitter app and generate new credentials, However, you can search for the TraceBack error returned ? – ZdaR Jun 01 '15 at 16:27
  • Well the funny thing was that I wasn't getting any kind of error returned - just nothing coming through the stream. Even more weirdly, this problem seems to have resolved itself today as the script is working fine! Thanks for your help. – Cian Murphy Jun 02 '15 at 10:23

0 Answers0