1

Is anyone able to explain how to make Ghost.py work with a proxy? I've checked out the code but it's not there.

iChux
  • 2,266
  • 22
  • 37
  • do you have any update on `set_proxy`? It remains very unclear on my side. If you want to specify one, I do something like `ghost=Ghost(wait_timeout=20)` then `ghost.set_proxy(type_='http', host="http://myproxy.net", port=7676)` but `page, res = ghost.open()` gives a `page=None` result – Colonel Beauvel Feb 06 '15 at 10:59
  • @ColonelBeauvel Are you sure of the proxy and it's port, to know that they are working? – iChux Feb 11 '15 at 11:45
  • Yes completely sure since they are working when I use `urllib2` to scrap some static data with proxy. So the proxy and port are correct, I guess I am doing something wrong with `ghost` but do not know what ... – Colonel Beauvel Feb 11 '15 at 12:42
  • @ColonelBeauvel I don't have any working proxy to test it out now. I would do that as soon as I'm able to and then get back to you with the results. Sorry for the inconvenience. – iChux Feb 11 '15 at 13:22

4 Answers4

3

I've found it in the ghost.py file. They did a very good job in it. It's a method on line 835, as set_proxy(). It's just how to use it that I'm yet to try out:

def set_proxy(self, type_, host='localhost', port=8888, user='',
        password=''):
    """Set up proxy for FURTHER connections.

    :param type_: proxy type to use: \
        none/default/socks5/https/http.
    :param host: proxy server ip or host name.
    :param port: proxy port.
    """
    _types = {
        'default': QNetworkProxy.DefaultProxy,
        'none': QNetworkProxy.NoProxy,
        'socks5': QNetworkProxy.Socks5Proxy,
        'https': QNetworkProxy.HttpProxy,
        'http': QNetworkProxy.HttpCachingProxy
    }

    if type_ is None:
        type_ = 'none'
    type_ = type_.lower()
    if type_ in ['none', 'default']:
        self.manager.setProxy(QNetworkProxy(_types[type_]))
        return
    elif type_ in _types:
        proxy = QNetworkProxy(_types[type_], hostName=host, port=port,
            user=user, password=password)
        self.manager.setProxy(proxy)
    else:
        raise ValueError('Unsupported proxy type:' + type_ \
        + '\nsupported types are: none/socks5/http/https/default')

What I don't understand now is what "QNetworkProxy.DefaultProxy" means. It's said to be the default proxy. So, what's the default proxy?

iChux
  • 2,266
  • 22
  • 37
2

If in Ghost.py the way of making tcp connections based on Qt api, then you may use Qt/PySide api, see QNetworkProxy::setApplicationProxy(). Otherwise, if Ghost.py not using Qt Api, but for example curl libe, then you try to set environment variable "http_proxy"

yshurik
  • 772
  • 5
  • 12
2

Documentation says that QNetworkProxy.DefaultProxy: Proxy is determined based on the application proxy set using setApplicationProxy() So if proxy is set by QNetworkProxy::setApplicationProxy(), then the call set_proxy('default') will make to use it (it will pass the proxy to self.manager, which I guess is the QNetworkAccessManager object).

yshurik
  • 772
  • 5
  • 12
  • 1
    So, does this mean that if I set an https proxy without specifying that the QNetworkAccessManager will understand that its https? – iChux Jan 23 '14 at 10:38
2

You can use below code. It works for me,

from ghost import Ghost, Session
ghost = Ghost()
with ghost.start():
    session = Session(ghost)
    session.wait_timeout = 999
    session.set_proxy('http', str(ip), int(port), str(username), str(password))
    page, resource = session.open(url)
    print session.content # Prints html content
    print page.headers, page.url, page.http_status

The ghost object has only one method i.e start(). Rest of them are defined as methods of Session class.

theBuzzyCoder
  • 2,652
  • 2
  • 31
  • 26