Creating facebook webproxy with python

Question

I'm trying to create webproxy for personal use to access facebook (it is often blocked from a few locations where I happen to spend some time).

Started from this code: http://code.google.com/p/gevent/source/browse/examples/webproxy.py?name=1.0b2

I've modified it to work with the cookies replacing urllib with requests in the following way:

def proxy_post(path, env, proxy_url, start_response):
    if '://' not in path:
        path = 'http://' + path
    try:
        #response = br.submit(path, env)
        response = requests.post(path, params = env)
        print '%s: %s' % (path, response)
        headers = [(k, v) for (k, v) in response.headers.items() if k not in drop_headers]
        scheme, netloc, path, params, query, fragment = urlparse(path)
        host = (scheme or 'http') + '://' + netloc
    except Exception, ex:
        sys.stderr.write('error while reading %s:\n' % path)
        traceback.print_exc()
        tb = traceback.format_exc()
        error_str = escape(str(ex) or ex.__class__.__name__ or 'Error')
        return ['<h1>%s</h1><h2>%s</h2><pre>%s</pre>' % (error_str, escape(path), escape(tb))]
    else:
        start_response('%s OK' % response.status_code, headers)
        data = response.content
        data = fix_links(data, proxy_url, host)
        return [data]

But when I try to login I've got the following error:

Traceback (most recent call last):
  File "/usr/local/lib/python2.7/dist-packages/gevent/wsgi.py", line 116, in handle
    self.data.extend(result)
TypeError: 'NoneType' object is not iterable
<WSGIServer fileno=3 address=0.0.0.0:8088>: Failed to handle request:
  request = <http_request "POST /https://www.facebook.com/login.php?login_attempt=1 HTTP/1.1" 127.0.0.1:56381>
  application = <function application at 0x2be2578>

127.0.0.1 - - [2013-04-13 20:08:03] "POST /https://www.facebook.com/login.php?login_attempt=1 HTTP/1.1" 500 21 "http://127.0.0.1:8088/http://www.facebook.com" "Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0"

My guess with happens because of some problem while creating WSGI response but I could figure out exactly why.

Why don't you just install Squid? – utapyngo Apr 14 '13 at 00:54 — utapyngo, Apr 14 '13 at 00:54
@utapyngo this is a great idea, thank you. – Moonwalker Apr 14 '13 at 12:13 — Moonwalker, Apr 14 '13 at 12:13

running.t · Answer 1 · 2013-04-12T10:45:18.987

0

I took a look at the webproxy you have mentioned. Inside def application(env, start_response): there is following code:

 elif (method, path) == ('POST', ''):
    key, value = env['wsgi.input'].read().strip().split('=')
    assert key == 'url', repr(key)
    start_response('302 Found', [('Location', join(proxy_url, unquote(value)))])
elif method == 'POST':
    start_response('404 Not Found', [])

So if method is POST and path is not empty 404 is returned. According to response you attached you are using POST method with path /https://www.facebook.com/login.php?login_attempt=1 I would start searching there. That might have sth in common with differences between results returned by urllib2.open() and mechanize.Browser().open().

Side note: sth is wrong witht that path, it should not start with /https://www.facebook.com/.

edited Apr 12 '13 at 10:45

answered Apr 12 '13 at 10:39

running.t

5,329
3
32
50

Thank you, it really helped. Now I've created a custom handler for the post requests but this is not working either. Could you please take a look and the new version of the code? – Moonwalker Apr 13 '13 at 16:14
Now it is even less readable. But still I can see that you are trying to post `http://127.0.0.1:8088/http://www.facebook.com` what is obviously wrong. – running.t Apr 15 '13 at 10:19

Creating facebook webproxy with python

1 Answers1