Stack Overflow
Stack Overflow

Questions tagged [urllib]

Python module providing a high-level interface for fetching data across the World Wide Web. Predecessor to urllib2. In Python 3, urllib2 and urllib have been reorganized and merged into urllib.

For Python 2, the urllib module is the predecessor of the urllib2 module, while the latter still uses some functionality of the former.

For Python 3, the urllib package was reorganized. It now has no content of its own. All methods and classes are in several submodules:

Note that doesn't exist in Python 3 anymore.

3960 questions
71
votes
5 answers

Python - make a POST request using Python 3 urllib

I am trying to make a POST request to the following page: http://search.cpsa.ca/PhysicianSearch In order to simulate clicking the 'Search' button without filling out any of the form, which adds data to the page. I got the POST header information by…
Daniel Paczuski Bak
  • 3,720
  • 8
  • 32
  • 78
68
votes
7 answers

How to handle response encoding from urllib.request.urlopen() , to avoid TypeError: can't use a string pattern on a bytes-like object

I'm trying to open a webpage using urllib.request.urlopen() then search it with regular expressions, but that gives the following error: TypeError: can't use a string pattern on a bytes-like object I understand why, urllib.request.urlopen()…
kryptobs2000
  • 3,289
  • 3
  • 27
  • 30
68
votes
2 answers

Handling urllib2's timeout? - Python

I'm using the timeout parameter within the urllib2's urlopen. urllib2.urlopen('http://www.example.org', timeout=1) How do I tell Python that if the timeout expires a custom error should be raised? Any ideas?
RadiantHex
  • 24,907
  • 47
  • 148
  • 244
67
votes
3 answers

AttributeError: module 'urllib' has no attribute 'parse'

python 3.5.2 code 1 import urllib s = urllib.parse.quote('"') print(s) it gave this error: AttributeError: module 'urllib' has no attribute 'parse' code 2 from urllib.parse import quote # import urllib # s = urllib.parse.quote('"') s =…
Hong Yinjie
  • 671
  • 1
  • 5
  • 3
65
votes
10 answers

Python 3.5.1 urllib has no attribute request

I have tried import urllib.request or import urllib The path for my urllib is /Library/Frameworks/Python.framework/Versions/3.5/lib/python3.5/urllib/__init__.py I am wondering where is urlopen, or is my python module pointing to the wrong file?
user1999806
  • 681
  • 1
  • 5
  • 10
61
votes
7 answers

Making a POST call instead of GET using urllib2

There's a lot of stuff out there on urllib2 and POST calls, but I'm stuck on a problem. I'm trying to do a simple POST call to a service: url = 'http://myserver/post_service' data = urllib.urlencode({'name' : 'joe', 'age' :…
alfonso.kim
  • 2,844
  • 4
  • 32
  • 37
57
votes
12 answers

Get size of a file before downloading in Python

I'm downloading an entire directory from a web server. It works OK, but I can't figure how to get the file size before download to compare if it was updated on the server or not. Can this be done as if I was downloading the file from a FTP…
PabloG
  • 25,761
  • 10
  • 46
  • 59
57
votes
6 answers

Python urllib vs httplib?

When would someone use httplib and when urllib? What are the differences? I think I ready urllib uses httplib, I am planning to make an app that will need to make http request and so far I only used httplib.HTTPConnection in python for requests, and…
jahmax
  • 8,181
  • 7
  • 26
  • 25
57
votes
1 answer

Python3: JSON POST Request WITHOUT requests library

I want to send JSON encoded data to a server using only native Python libraries. I love requests but I simply can't use it because I can't use it on the machine which runs the script. I need to do it without. newConditions = {"con1":40, "con2":20,…
moritzg
  • 4,266
  • 3
  • 37
  • 62
56
votes
8 answers

How to know if urllib.urlretrieve succeeds?

urllib.urlretrieve returns silently even if the file doesn't exist on the remote http server, it just saves a html page to the named file. For example: urllib.urlretrieve('http://google.com/abc.jpg', 'abc.jpg') just returns silently, even if…
btw0
  • 3,516
  • 5
  • 34
  • 36
56
votes
4 answers

Python; urllib error: AttributeError: 'bytes' object has no attribute 'read'

Note: This is Python 3, there is no urllib2. Also, I've tried using json.loads(), and I get this error: TypeError: can't use a string pattern on a bytes-like object I get this error if I use json.loads() and remove the .read() from…
Parseltongue
  • 11,157
  • 30
  • 95
  • 160
54
votes
3 answers

Python 3 urllib produces TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str

I am trying to convert working Python 2.7 code into Python 3 code and I am receiving a type error from the urllib request module. I used the inbuilt 2to3 Python tool to convert the below working urllib and urllib2 Python 2.7 code: import…
Greg
  • 8,175
  • 16
  • 72
  • 125
54
votes
5 answers

How to unquote a urlencoded unicode string in python?

I have a unicode string like "Tanım" which is encoded as "Tan%u0131m" somehow. How can i convert this encoded string back to original unicode. Apparently urllib.unquote does not support unicode.
hamdiakoguz
  • 15,795
  • 9
  • 33
  • 27
53
votes
6 answers

Replace special characters in a string in Python

I am using urllib to get a string of html from a website and need to put each word in the html document into a list. Here is the code I have so far. I keep getting an error. I have also copied the error below. import urllib.request url =…
user2363217
  • 695
  • 1
  • 7
  • 15
51
votes
6 answers

Python: Get HTTP headers from urllib2.urlopen call?

Does urllib2 fetch the whole page when a urlopen call is made? I'd like to just read the HTTP response header without getting the page. It looks like urllib2 opens the HTTP connection and then subsequently gets the actual HTML page... or does it…
shigeta
  • 1,791
  • 3
  • 19
  • 32
1 2
3
…
99 100