Stack Overflow
Stack Overflow

Questions tagged [httplib]

A module from Python standard lib that defines classes which implement the client side of the HTTP and HTTPS protocols.

Official Python documentation at http://docs.python.org/2/library/httplib.html

328 questions
19
votes
4 answers

How to detect when a site check redirects to another page using the requests module?

For example, if I go to www.yahoo.com/thispage, and yahoo has set up a filter to redirect /thispage to /thatpage. So whenever someone goes to /thispage, they will land on /thatpage. If I use httplib/requests/urllib, will it know that there was a…
iCodeLikeImDrunk
  • 17,085
  • 35
  • 108
  • 169
19
votes
3 answers

Tunneling httplib Through a Proxy

I am trying to figure out how to send data to a server through a proxy. I was hoping this would be possible through tor but being as tor uses SOCKS it apparently isn't possible with httplib (correct me if I am wrong) This is what I have right…
Max00355
  • 827
  • 2
  • 12
  • 28
18
votes
2 answers

In requests library, how can I avoid "HttpConnectionPool is full, discarding connection" warning?

I'm using python requests library with sessions: def _get_session(self): if not self.session: self.session = requests.Session() return self.session And sometimes I'm getting this warning in my logs: [2014/May/12 14:40:04 WARNING ]…
mnowotka
  • 16,430
  • 18
  • 88
  • 134
17
votes
4 answers

Python script to see if a web page exists without downloading the whole page?

I'm trying to write a script to test for the existence of a web page, would be nice if it would check without downloading the whole page. This is my jumping off point, I've seen multiple examples use httplib in the same way, however, every site I…
some1
  • 2,447
  • 8
  • 26
  • 23
17
votes
3 answers

httplib CannotSendRequest error in WSGI

I've used two different python oauth libraries with Django to authenticate with twitter. The setup is on apache with WSGI. When I restart the server everything works great for about 10 minutes and then the httplib seems to lock up (see the…
Dave
  • 1,658
  • 3
  • 17
  • 19
14
votes
2 answers

Why is the block size for Python httplib's reads hard coded as 8192 bytes

I'm looking to make a fast streaming download -> upload to move large files via HTTP from one server to another. During this, I've noticed that httplib, that is used by urllib3 and therefore also requests, seems to hard code how much it fetches from…
Michal Charemza
  • 25,940
  • 14
  • 98
  • 165
13
votes
4 answers

python httplib/urllib get filename

is there a possibillity to get the filename e.g. xyz.com/blafoo/showall.html if you work with urllib or httplib? so that i can save the file under the filename on the server? if you go to sites like xyz.com/blafoo/ you cant see the…
HappyHacking
  • 878
  • 1
  • 12
  • 28
11
votes
1 answer

debugging connection with urllib2+httplib.debuglevel sometimes not showing debug info

Trying to get a login script working, I kept getting the same login page returned, so I turned on debugging of the http stream (can't use wireshark or the like because of https). I got nothing, so I copied the example, it works. Any query to…
snoonan
  • 160
  • 1
  • 5
10
votes
3 answers

Selenium headless browser webdriver [Errno 104] Connection reset by peer

I am trying to scrape data from the URLs below. But selenium fails when driver.get(url) Some times the error is [Errno 104] Connection reset by peer, sometimes [Errno 111] Connection refused. On rare days it works just fine and on my mac with real…
kumar
  • 2,570
  • 2
  • 17
  • 18
9
votes
4 answers

python httplib Name or service not known

I'm trying to use httplib to send credit card information to authorize.net. When i try to post the request, I get the following traceback: File "./lib/cgi_app.py", line 139, in run res = method() File "/var/www/html/index.py", line 113, in…
Chris
  • 2,619
  • 6
  • 27
  • 34
9
votes
1 answer

python: httplib.CannotSendRequest when nesting threaded SimpleXMLRPCServers

I am intermittently receiving a httplib.CannotSendRequest exception when using a chain of SimpleXMLRPCServers that use the SocketServer.ThreadingMixin. What I mean by 'chain' is the following: I have a client script which uses xmlrpclib to call a…
user1848244
  • 425
  • 3
  • 12
8
votes
2 answers

Url encode using python 2.7

>>> import httplib >>> x = httplib.HTTPConnection('localhost', 8080) >>> x.connect() >>> x.request('GET','/camera/store?fn=aaa&ts='+str.encode('2015-06-15T14:45:21.982600+00:00','ascii')+'&cam=ddd') >>> y=x.getresponse() >>> z=y.read() >>>…
Kevin Kai
  • 139
  • 1
  • 1
  • 12
8
votes
2 answers

How do I respond to a "CONNECT" method request in a proxy server using socket in python?

I am currently programming a proxy server using httplib, and when I try to connect to HTTPS websites (such as facebook and google) my client sends me "CONNECT" requests that look like this: CONNECT www.google.co.il:443 HTTP/1.1\r\n User-Agent:…
yoloz
  • 83
  • 1
  • 1
  • 6
8
votes
1 answer

How to POST chunked encoded data in Python

I am trying to POST chunked encoded data to httpbin.org/post. I tried two options: Requests and httplib Using Requests #!/usr/bin/env python import requests def gen(): l = range(130) for i in l: yield '%d' % i if…
Piyush Kansal
  • 1,201
  • 4
  • 18
  • 26
8
votes
1 answer

Handling IncompleteRead,URLError

it's a piece of web mining script. def printer(q,missing): while 1: tmpurl=q.get() try: image=urllib2.urlopen(tmpurl).read() except httplib.HTTPException: missing.put(tmpurl) continue …
from __future__
  • 305
  • 2
  • 6
1
2
3
…
21 22