Http Request through proxy in python having @ in password

Question

How to escape @ character in the password of proxy. So that python can create the request correctly. I have tried \\ but still not able to hit the url correctly.

 proxy = {
        "http": "http://UserName:PassWord@X.X.X.X:Port_No"
    }

Update question:

I am using python requests module for the http request. It split the string (to get host) from first occurrence of @ where as it was suppose to split from second @.

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 55, in get
    return request('get', url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/api.py", line 44, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 335, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/sessions.py", line 438, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/requests/adapters.py", line 327, in send
    raise ConnectionError(e)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='XXXXXXXX@X.X.X.X', port=XXXXX): Max retries exceeded with url: http:/URL (Caused by <class 'socket.gaierror'>: [Errno -2] Name or service not known)

one in th password and second is the seperator of password and ip i.e. 'username:st@ckoverflow@192.168.1.200:12345' — Naresh, May 19 '15 at 05:58
Huh? I don't see that anywhere in your question. Regardless, this proxy works for me too: `http://username:st@ckoverflow@127.0.0.1:12345`. And have you tried `http://username:st%40ckoverflow@127.0.0.1:12345` yet? — mhawke, May 19 '15 at 06:13

score 1 · Answer 1 · edited May 23 '17 at 10:26

1

You have to do urlencoding like in this post:

Escaping username characters in basic auth URLs

This way the @ in the PW becomes %40

edited May 23 '17 at 10:26

Community

1
1

answered May 18 '15 at 10:05

jhinghaus

770
10
27

mhawke · Answer 2 · 2015-05-18T12:19:34.190

-1

You don't mention which library you are using to perform your HTTP requests, so you should consider using requests, not only to solve this problem, but because it is a great library.

Here is how to use a proxy with basic authentication:

import requests

proxy = {'http': 'http://UserName:PassWord@X.X.X.X:Port_No'}
r = requests.get("http://whereever.com", proxies=proxy)

Update

Successfully tested with requests and proxy URLs:

http://UserName:PassWord@127.0.0.1:1234
http://UserName:PassWord@@127.0.0.1:1234
http://User@Name:PassWord@1234@127.0.0.1:1234

If, instead, you need to use Python's urllib2 library, you can do this:

import urllib2

handler = urllib2.ProxyHandler({'http': 'http://UserName:PassWord@X.X.X.X:Port_No'})
opener = urllib2.build_opener(handler)
r = opener.open('http://whereever.com')

Note that in neither case is it necessary to escape the @.

A third option is to set environment variables HTTP_PROXY and/or HTTPS_PROXY (in *nix).

edited May 18 '15 at 12:19

answered May 18 '15 at 10:50

mhawke

84,695
9
117
138

I have updated the question and i do get the error when i use request module. And it works fine if i change the password without @ – Naresh May 18 '15 at 11:47
@Naresh: OK, I see that I have misunderstood the problem which is that the `@` is contained within the password field. Did you try to urlencode the URL as suggested by jhinghaus? – mhawke May 18 '15 at 12:03
@Naresh: actually, it does work with a `@` in the password, i.e. 2 `@` characters such as `http://UserName:PassWord@1234@127.0.0.1:1234` and `http://User@Name:PassWord@1234@127.0.0.1:1234` etc. I am using requests version 1.2.3. What version are you using? – mhawke May 18 '15 at 12:17

Http Request through proxy in python having @ in password

2 Answers2