Unknown error in sixohsix's Twitter API for Python

Question

I have a small script that repeatedly (hourly) fetches tweets from the API, using sixohsix's Twitter Wrapper for Python. I am successful with handling most, if not all of the errors coming from the Twitter API, i.e. all the 5xx and 4xx stuff.

Nonetheless I randomly observe the below error traceback (only once in 2-3 days). I mean the program exits and the traceback is displayed in the shell. I have no clue what this could mean, but think it is not directly related to what my script does since it has proved itself to correctly run most of the time.

This is where I call a function of the wrapper in my script:

KW = {
        'count': 200, # number of tweets to fetch (fetch maximum)
        'user_id' : tweeter['user_id'],
        'include_rts': 'false', # do not include native RT's 
        'trim_user' : 'true', 
     }
     timeline = tw.twitter_request(tw_endpoint,\
         tw_endpoint.statuses.user_timeline, KW)

The function tw.twitter_request(tw_endpoint, tw_endpoint.statuses.user_timeline, KW) basically does return tw_endpoint.statuses_user_timeline(**args), where args translate to KW, and tw_endpoint is an OAuthorized endpoint gained from using the sixohsix's library's

return twitter.Twitter(domain='api.twitter.com', api_version='1.1',
                    auth=twitter.oauth.OAuth(access_token, access_token_secret,
                    consumer_key, consumer_secret))

This is the traceback:

Traceback (most recent call last):
  File "search_twitter_entities.py", line 166, in <module>
    tw_endpoint.statuses.user_timeline, KW)
  File "/home/tg/mild/twitter_utils.py", line 171, in twitter_request
    return twitter_function(**args)
  File "build/bdist.linux-x86_64/egg/twitter/api.py", line 173, in __call__
  File "build/bdist.linux-x86_64/egg/twitter/api.py", line 177, in _handle_response
  File "/usr/lib/python2.7/urllib2.py", line 126, in urlopen
    return _opener.open(url, data, timeout)
  File "/usr/lib/python2.7/urllib2.py", line 400, in open
    response = self._open(req, data)
  File "/usr/lib/python2.7/urllib2.py", line 418, in _open
    '_open', req)
  File "/usr/lib/python2.7/urllib2.py", line 378, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.7/urllib2.py", line 1215, in https_open
    return self.do_open(httplib.HTTPSConnection, req)
  File "/usr/lib/python2.7/urllib2.py", line 1180, in do_open
    r = h.getresponse(buffering=True)
  File "/usr/lib/python2.7/httplib.py", line 1030, in getresponse
    response.begin()
  File "/usr/lib/python2.7/httplib.py", line 407, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python2.7/httplib.py", line 371, in _read_status
    raise BadStatusLine(line)
httplib.BadStatusLine: ''

The only thing I can gain from that traceback is that the error happens somewhere deep inside another Python library and has something to do with an invalid HTTP stat coming from the Twitter API or the wrapper... But as I said, maybe some of you could give me a hint on how to debug/solve this since it is pretty annoying having to regularly check my script and restart it to continue fetching tweets.

EDIT: To clarify this a little, the first two functions in the traceback are already in a try-except block. For example, the try-except-Block in File "twitter_utils.py" filters out 40x and 50x exceptions, but also looks for general exceptions with only except:. So what I don't understand is why the error is not getting caught at this position and instead, the program is force-closed and a traceback printed? Shortly speaking I am in the situation where I cannot catch an error, just like a parse error in a PHP script. So how would I do this?

score 0 · Answer 1 · answered Jan 25 '13 at 04:20

Perhaps this will point you in the right direction. This is what's being called when BadStatusLine is called upon:

class BadStatusLine(HTTPException):
    def __init__(self, line):
        if not line:
            line = repr(line)
        self.args = line,
        self.line = line

I'm not too familiar with httplib, but if I had to guess, you're geting an empty response/error line and, well, it can't be parsed. There are comments before the line you're program is stopping at:

    # Presumably, the server closed the connection before
    # sending a valid response.
    raise BadStatusLine(line)

If twitter is closing the connection before sending a response, you could try again, meaning do a try/except at "search_twitter_entities.py", line 166 a couple times (ugly).

try:
     timeline = tw.twitter_request(tw_endpoint,\
              tw_endpoint.statuses.user_timeline, KW)
except:
     try:
          timeline = tw.twitter_request(tw_endpoint,\
                   tw_endpoint.statuses.user_timeline, KW) # try again
     except:
          pass

Or, assuming you can reassign timeline as none each time, do a while loop:

timeline = None
while timeline == None:
     try:
          timeline = tw.twitter_request(tw_endpoint,\
              tw_endpoint.statuses.user_timeline, KW)
     except:
          pass

I didn't test of of that. Check for bad code.

Well, it's actually not that simple. First of all, `timeline = tw.twitter_request(tw_endpoint,\ tw_endpoint.statuses.user_timeline, KW)` already IS in a try-except block, also including one `except:` that accepts all exceptions and thus should catch all of them. The same with the function `twitter_request`. Still, no exception is thrown and the program exits immediatly with the traceback. So the actual question is: How to catch that error without terminating the script? — grssnbchr, Jan 31 '13 at 12:25

Unknown error in sixohsix's Twitter API for Python

1 Answers1