Stack Overflow
Stack Overflow

Questions tagged [urlopen]

The urlopen is a method of the urllib library in Python, used to open a particular URL.

The urlopen is a method of the urllib library in Python, used to open a particular URL. As a result, a file-like object is returned that contains information about the URL - headers, response data and other details about the requested URL resource.

369 questions
13
votes
4 answers

Passing a variable in url?

So I'm new in python and I desperately need help. I have a file which has a bunch of ids (integer values) written in 'em. Its a text file. Now I need to pass each id inside the file into a url. For example "https://example.com/[id]" It will be…
user1452759
  • 8,810
  • 15
  • 42
  • 58
12
votes
5 answers

Why urllib.urlopen.read() does not correspond to source code?

I'm trying to fetch the following webpage: import urllib urllib.urlopen("http://www.gallimard-jeunesse.fr/searchjeunesse/advanced/(order)/author?catalog[0]=1&SearchAction=1").read() The result does not correspond to what I see when inspecting the…
Nikolay Nikolov
  • 131
  • 1
  • 4
10
votes
2 answers

Urllib.urlopen() works on SSLv3 urls with Python 2.6.6 on 1 machine, but not with 2.6.7/2.7.2 on another

Spent a good part of a day on this, and I'm realllly at my wit's end. I have 1 machine "A" with Python 2.6.6/2.7.2 installed, and another machine "B" with Python 2.6.7/2.7.2 installed. On machine A, I can get a SSLv3-encrypted website with…
jwayne
  • 706
  • 2
  • 7
  • 20
10
votes
4 answers

Python selenium test gets stuck in urlopen

My app relies on: Python 3 Django 1.8 Weasyprint Selenium It runs flawlessly on dev and production environment, but not while testing with selenium. Using weasyprint, I create a PDF from HTML, this library uses urllib to download CSS (e.g.…
SebCorbin
  • 1,645
  • 1
  • 12
  • 23
10
votes
1 answer

Urllib's urlopen breaking on some sites (e.g. StackApps api): returns garbage results

I'm using urllib2's urlopen function to try and get a JSON result from the StackOverflow api. The code I'm using: >>> import urllib2 >>> conn = urllib2.urlopen("http://api.stackoverflow.com/0.8/users/") >>> conn.readline() The result I'm…
Edan Maor
  • 9,772
  • 17
  • 62
  • 92
9
votes
2 answers

Passing a list as a url value to urlopen

Motivation Motivated by this problem - the OP was using urlopen() and accidentally passed a sys.argv list instead of a string as a url. This error message was thrown: AttributeError: 'list' object has no attribute 'timeout' Because of the way…
alecxe
  • 462,703
  • 120
  • 1,088
  • 1,195
9
votes
2 answers

Again urllib.error.HTTPError: HTTP Error 400: Bad Request

Hy! I tried to open web-page, that is normally opening in browser, but python just swears and does not want to work. import urllib.request, urllib.error f =…
Wanu
  • 93
  • 1
  • 1
  • 5
8
votes
2 answers

TypeError: urlopen() got multiple values for keyword argument 'body' while executing tests through Selenium and Python on Kubuntu 14.04

im trying to run a selenium in python on Kubuntu 14.04. I get this error message trying with chromedriver or geckodriver, both same error. Traceback (most recent call last): File "vse.py", line 15, in driver =…
Geeh Oliveira
  • 103
  • 2
  • 8
7
votes
1 answer

urlopen error [Errno 111] Connection refused for selenium python

When I try to log out from the testing page, there occurs an connection refused error. This is the command / code I've used: driver.find_element_by_xpath("//a[@href = '/logged/pages/user/logout.php']").click() I also tried by linktext, or…
xiaoyan zhang
  • 153
  • 1
  • 9
6
votes
0 answers

python3 requests "freezes" but urllib.request.urlopen does not?

I ran into a strange behavior of the requests package ... or so I believe: (1) for a certain website, requests.get runs on Windows, but it freezes on Linux (or raises timeout error if timeout is set to several seconds) (2) urllib.request.urlopen…
Olaf
  • 145
  • 1
  • 1
  • 12
6
votes
1 answer

urllib.request.urlopen return bytes, but I cannot decode it

I tried parsing a web page using urllib.request's urlopen() method, like: from urllib.request import Request, urlopen req = Request(url) html = urlopen(req).read() However, the last line returned the result in bytes. So I tried decoding it,…
Blaszard
  • 30,954
  • 51
  • 153
  • 233
6
votes
4 answers

Python: Urllib.urlopen nonnumeric port

for the following code theurl = "https://%s:%s@members.dyndns.org/nic/update?hostname=%s&myip=%s&wildcard=NOCHG&mx=NOCHG&backmx=NOCHG" % (username, password, hostname, theip) conn = urlopen(theurl) # send the request to the url print(conn.read()) …
user404288
  • 61
  • 1
  • 2
5
votes
1 answer

How to use python urlopen scraping after a page finish loading all searching result?

I am trying to scrape air ticket info(including plane info and price info, etc.) from http://flight.qunar.com/ using python3 and BeautifulSoup. Below is the python code I am using. In this code I tried to scrape flight info from Beijing(北京) to…
Sam Wei
  • 99
  • 4
  • 11
4
votes
1 answer

why python gevent is slower than serial ?

i am using gevent to preform concurrent download. based on this example this is the code: import gevent from gevent import monkey urls =…
yossi
  • 12,945
  • 28
  • 84
  • 110
4
votes
3 answers

Web scraping urlopen in python

I am trying to get the data from this website: http://www.boursorama.com/includes/cours/last_transactions.phtml?symbole=1xEURUS It seems like urlopen don't get the html code and I don't understand why. It goes like: html =…
user906780
1
2
3
…
24 25