Stack Overflow
Stack Overflow

Questions tagged [urllib3]

`urllib3` is a 3rd-party Python HTTP library with thread-safe connection pooling, file post, and more.

urllib3 is a library for python 2 and 3 providing a HTTP API with thread-safe connection pooling, file posting support, redirection handling and retries.

Documentation can be found at https://urllib3.readthedocs.io/en/latest/

760 questions
5
votes
1 answer

Disable HeaderParsingError appearing from python3 urllib3

how can I suppress the Failed to parse headers error that appears from the urllib3 library? The following errors keep appearing: Failed to parse headers (url=https://test): [StartBoundaryNotFoundDefect(), MultipartInvariantViolationDefect()],…
Derek Lee
  • 475
  • 1
  • 6
  • 20
5
votes
1 answer

In urllib3, HTTP request hangs ... but doesn't in curl

(Thanks for taking a look at this!) I'm trying to use python3 and simple urllib3 http.request to read HTML from https://login.morganstanleyclientserv.com. It seems like the server is resetting the connection, and eventually urllib3's retries give…
bdemchak
  • 263
  • 2
  • 10
5
votes
1 answer

How do you use cookiejar with urllib3?

How do you use cookiejar with urllib3? According to Python urllib3 and how to handle cookie support? from 2010 it wasn't supported then. However, in the change log for urllib3 1.22 (2017-07-20) there's mention of a compatibility fix for…
Uyghur Lives Matter
  • 18,820
  • 42
  • 108
  • 144
5
votes
1 answer

Do I need to close a urllib.request.urlopen connection in Python?

I'm working with the urllib.request module in Python 3. My question is: Do we need to close the connection if I open it using urllib.request.ulropen?
A. Khoja
  • 57
  • 1
  • 5
5
votes
1 answer

counting words inside a webpage

I need to count words that are inside a webpage using python3. Which module should I use? urllib? Here is my Code: def web(): f =("urllib.request.urlopen("https://americancivilwar.com/north/lincoln.html") lu = f.read() print(lu)
birajad
  • 482
  • 1
  • 8
  • 16
5
votes
1 answer

How to fix urllib3 RuntimeError: Requests dependency 'urllib3' must be version >= 1.21.1, < 1.22?

I am a new developer and learning to code in Python 3.4.2. I am running Debian linux on a Raspberry Pi3. After the fresh install I did both sudo apt-get update and sudo apt-get upgrade to get everything up to date. I am trying to test a section…
DevScientist
  • 51
  • 1
  • 3
5
votes
1 answer

What is the practical difference between these two ways of making web connections in Python?

I have notice there are several ways to iniciate http connections for web scraping. I am not sure if some are more recent and up-to-date ways of coding, or if they are just different modules with different advantages and disadvantages. More…
Victor Domingos
  • 1,003
  • 1
  • 18
  • 40
5
votes
1 answer

Python requests - how to add multiple own certificates

Is there a way to tell the requests lib to add multiple certificates like all .pem files from a specified folder? import requests, glob CERTIFICATES = glob('/certs/') url = '127.0.0.1:8080' requests.get(url, cert=CERTIFICATES) Seems to work only…
PlagTag
  • 6,107
  • 6
  • 36
  • 48
5
votes
1 answer

urllib3 download a file using specified user agent

What is the correct way to update the user agent information in urllib3? How can I check that the user agent information was indeed changed and is being used? For example: user_agent = {'user-agent': 'Mozilla/5.0 (Windows NT 6.3; rv:36.0)…
jmunsch
  • 22,771
  • 11
  • 93
  • 114
5
votes
1 answer

AttributeError: 'NoneType' object has no attribute 'wrap_socket' in GAE

I am try to fetch some tweets in Google App Engine and doing some analysis on that tweets. Due to some issue in urllib3, I am facing the following error : AttributeError: 'NoneType' object has no attribute 'wrap_socket' The last three call are…
krishna Prasad
  • 3,541
  • 1
  • 34
  • 44
5
votes
1 answer

Why HTTPSConnectionPool doesn't work when PoolManager does?

I have tested a 'POST' request with both PoolManager and HTTPSConnectionPool. The first one works, the other throw me a : urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='https://some.url.com', port=443): Max retries exceeded with url:…
Yann
  • 134
  • 1
  • 11
5
votes
1 answer

urllib3 on python 2.7 SNI error on Google App Engine

I'm trying to download an HTTPS page from my site hosted on Google App Engine with SNI. No matter what library I use, I get the following error: [Errno 8] _ssl.c:504: EOF occurred in violation of protocol I've tried solving the error in many ways,…
Tzach
  • 12,889
  • 11
  • 68
  • 115
5
votes
1 answer

How do I pass raw POST data into urllib3?

Trying to use urllib3 to post JSON-encoded data. Just want my POST payload to be raw JSON string, with content type application/json. I just cannot see how to do this. The urllib3 documentation describes posting data in "fields", i.e. dicts with…
user2855982
  • 51
  • 1
  • 1
  • 2
5
votes
4 answers

Proxy connection with Python

I have been attempting to connect to URLs from python. I have tried: urllib2, urlib3, and requests. It is the same issue that i run up against in all cases. Once I get the answer I imagine all three of them would work fine. The issue is…
bevanj
  • 254
  • 1
  • 3
  • 12
4
votes
3 answers

How to get log of every retry attempt made by Python Requests Library?

I'm using following code to implement retry mechanism in python requests. from requests import Session from requests.adapters import HTTPAdapter from urllib3.util.retry import Retry s = Session() retries = Retry(total=5, …
Hassan Sabir
  • 60
  • 1
  • 9
1 2 3
…
50 51