Stack Overflow
Stack Overflow

Questions tagged [urlopen]

The urlopen is a method of the urllib library in Python, used to open a particular URL.

The urlopen is a method of the urllib library in Python, used to open a particular URL. As a result, a file-like object is returned that contains information about the URL - headers, response data and other details about the requested URL resource.

369 questions
4
votes
2 answers

DeadlineExceededError: ApplicationError: 5 in using urllib2.urlopen() function

In my application I am using the urllib2.urlopen() function to call an api and get the result from that api.But this is not working fine.Sometimes it shows the result but sometimes it gives the following error: Traceback (most recent call last): …
bitanalyzer
  • 124
  • 2
  • 10
4
votes
2 answers

urlib2.urlopen through proxy fails after a few calls

Edit: after much fiddling, it seems urlgrabber succeeds where urllib2 fails, even when telling it close the connection after each file. Seems like there might be something wrong with the way urllib2 handles proxies, or with the way I use it…
Nicolas Lefebvre
  • 4,247
  • 1
  • 25
  • 29
4
votes
2 answers

how to handle "getaddrinfo failed"?

Hallo, i have problem. i use mechanize, python 2.7 to connect some sites (the code is not important right now) i have list of sites and i connect to them one by now. When it happens the site from my list doesn't exist i get error:…
RogerZ
  • 61
  • 1
  • 2
  • 3
4
votes
2 answers

urlopen/requests.get not working in threads created in imported modules

I have a problem with urlopen (and requests.get) In my program, if I run it inside a thread (I tested with multiprocessing too) [update: a thread that has been created by an imported module] it won't run until the program ends. By "won't run" I mean…
René
  • 179
  • 8
4
votes
2 answers

How to crawl pagination pages? There is no url change when I Click next page

I use python3.5 and window10. When I crawl some pages, I usually used url changes using urlopen and 'for' iteration. like below code. from bs4 import BeautifulSoup import urllib f = open('Slave.txt','w') for i in range(1,42): html =…
김상엽
  • 41
  • 1
  • 2
4
votes
1 answer

Google App Engine: upload_data fails because "target machine actively refused it" on devserver

I'm trying to upload data from a CSV to my app using the devserver: appcfg.py upload_data --config_file="DataLoader.py" --filename="data.csv" --kind=Foo --url=http://localhost:8083/remote_api "path/to/app" The result: Application: appname; version:…
Nick Heiner
  • 119,074
  • 188
  • 476
  • 699
3
votes
1 answer

How do I set cookies using Python urlopen?

I am trying to fetch an html site using Python urlopen. I am getting this error: HTTPError: HTTP Error 302: The HTTP server returned a redirect error that would lead to an infinite loop The code: from urllib2 import Request request =…
yossi
  • 12,945
  • 28
  • 84
  • 110
3
votes
2 answers

Error while using urllib.request.urlopen in Python

What's wrong with this code? >>> from urllib.request import urlopen >>> for line in urlopen("http://google.com/"): print(line.decode("utf-8"))
Sergey
  • 47,222
  • 25
  • 87
  • 129
3
votes
1 answer

Mercurial and nginx. Error 10054

When I push files to the server, then after 60 seconds i get "urlopen error [Errno 10054]". I think it's because i push large files. I use nginx web-server with fast-cgi process. What nginx parametr responsible to that 60sec timeout? Or maybe i…
EthanHunt
  • 473
  • 2
  • 9
  • 19
3
votes
1 answer

How can I access a PDF file with Python through an automatic download link?

I am trying to create an automated Python script that goes to a webpage like this, finds the link at the bottom of the body text (anchor text "here"), and downloads the PDF that loads after clicking said download link. I am able to retrieve the HTML…
scraping_paint
  • 71
  • 5
3
votes
1 answer

Getting the error text from `urllib2.urlopen`

I used Python's urllib2.urlopen and got a 500 error from the server. How do I find the text of the error? I'm hoping that it has useful information.
Ram Rachum
  • 84,019
  • 84
  • 236
  • 374
3
votes
1 answer

Should I switch from "urllib.request.urlretrieve(..)" to "urllib.request.urlopen(..)"?

1. Deprecation problem In Python 3.7, I download a big file from a URL using the urllib.request.urlretrieve(..) function. In the documentation (https://docs.python.org/3/library/urllib.request.html) I read the following just above the…
K.Mulier
  • 8,069
  • 15
  • 79
  • 141
3
votes
1 answer

Python urllib freezes with specific URL

I am trying to fetch a page and urlopen hangs and never returns anything, although the web page is very light and can be opened with any browser without any problems import urllib.request with…
Kabal
  • 53
  • 6
3
votes
3 answers

BeautifulSoup HTTPResponse has no attribute encode

I'm trying to get beautifulsoup working with a URL, like the following: from urllib.request import urlopen from bs4 import BeautifulSoup html = urlopen("http://proxies.org") soup = BeautifulSoup(html.encode("utf-8"),…
Ke.
  • 2,484
  • 8
  • 40
  • 78
3
votes
1 answer

Python 2.7.13 does not respect NO_PROXY and makes urllib2.urlopen() error with "Tunnel connection failed: 403 Forbidden"

The following test passes with Python 2.7.11: def test_urllib(self): import urllib2 import ssl context = ssl._create_unverified_context() # service does not have a trusted cert url = urllib2.urlopen("https://intranet.adress:port/",…
user7368407
  • 41
  • 4
1 2
3
…
24 25