Python Proxy with Twisted

Question

Hello! I have this code:

from twisted.web import proxy, http
from twisted.internet import reactor

class akaProxy(proxy.Proxy):
    """
    Local proxy = bridge between browser and web application
    """

    def dataReceived(self, data):

        print "Received data..."

        headers = data.split("\n")
        request = headers[0].split(" ")

        method = request[0].lower()
        action = request[1]
        print action
        print "ended content manipulation"  
        return proxy.Proxy.dataReceived(self, data)

class ProxyFactory(http.HTTPFactory):
    protocol = akaProxy

def intercept(port):
    print "Intercept"
    try:                
        factory = ProxyFactory()
        reactor.listenTCP(port, factory)
        reactor.run()
    except Exception as excp:
        print str(excp)

intercept(1337)

I use above code to intercept everything between browser and web site. When using above, I configure my browser settings: to IP: 127.0.0.1 and Port: 1337. I put this script in remote server to act my remote server as proxy server. But when I change browser proxy IP settings to my server's it does not work. What I do wrong? What else I need to configure?

Answer 1

Presumably your dataReceived is raising an exception during its attempts to parse the data passed to it. Try enabling logging so you can see more of what's going on:

from twisted.python.log import startLogging
from sys import stdout
startLogging(stdout)

The reason your parser is likely to raise exceptions is that dataReceived is not called only with a complete request. It is called with whatever bytes are read from the TCP connection. This may be a complete request, a partial request, or even two requests (if pipelining is in use).

Answer 2

dataReceived in the Proxy context is handling "translation of rawData into lines", so it may be too early for trying your manipulation code. You can try overriding allContentReceived instead and you will have access to the complete headers and content. Here is an example that I believe does what you are after:

#!/usr/bin/env python
from twisted.web import proxy, http

class SnifferProxy(proxy.Proxy):
    """
    Local proxy = bridge between browser and web application
    """

    def allContentReceived(self):
        print "Received data..."
        print "method = %s" % self._command
        print "action = %s" % self._path
        print "ended content manipulation\n\n"
        return proxy.Proxy.allContentReceived(self)


class ProxyFactory(http.HTTPFactory):

    protocol = SnifferProxy

if __name__ == "__main__":
    from twisted.internet import reactor
    reactor.listenTCP(8080, ProxyFactory())
    reactor.run()