Stack Overflow
Stack Overflow

Questions tagged [urlopen]

The urlopen is a method of the urllib library in Python, used to open a particular URL.

The urlopen is a method of the urllib library in Python, used to open a particular URL. As a result, a file-like object is returned that contains information about the URL - headers, response data and other details about the requested URL resource.

369 questions
1
vote
2 answers

HTML data is hidden from urllib

How do I get the real content from this page: http://kursuskatalog.au.dk/da/course/74960/105E17-Demokrati-og-diktatur-i-komparativt-perspektiv All I get from the code below is some links to javascript and CSS files. Is there a way out of this?…
Kresten
  • 810
  • 13
  • 36
1
vote
2 answers

Posting data using urllib without SSL certificate validation

I need to post data to a REST interface, however the receiving host is using a self-signed certificate (and this is not about to change) so I need to ignore the obvious certificate validation error that I receive. My initial script looked like…
irwazr
  • 77
  • 1
  • 9
1
vote
1 answer

Read and put by chunk with urllib2.urlopen synchronously

I have a simple Python script which should read a file from HTTP source and make a PUT request to another HTTP source. block_size = 4096 file = urllib2.urlopen('http://path/to/someting.file').read(block_size) headers = {'X-Auth-Token': token_id,…
Semka Mesilov
  • 57
  • 2
  • 4
1
vote
2 answers

How can I download a file in Python3 with urlopen() or add custom headers to urlretrieve()?

tl;dr I want do download a file from a server who only allows certain User-Agents. I managed to get a 200 OK from the site by using following code: opener = urllib.request.build_opener() opener.addheaders = [('User-Agent', 'Interwebs Exploiter…
technical_difficulty
  • 427
  • 2
  • 5
  • 20
1
vote
1 answer

Access web site by location in python

Some web site have a pt., en. in the beginner or .br, .it at the end because of the server location. When I use the library of python as the function urlopen I have to pass the full adress string of the web site, including the termination string of…
hildogjr
  • 754
  • 2
  • 6
  • 17
1
vote
1 answer

Python urllib2.urlopen(url).read() is different from source code seen in Firefox

When I use urllib2.urlopen(url).read() I read a source code slightly different from what I read in Firefox. In source code seen in Firefox some special characters, such as quotation marks ("), apostrophe ('), etc are converted to %22, %27 etc. When…
Luigi
  • 11
  • 3
1
vote
0 answers

How can I convert a script for collecting data from a single URL to scrape hundreds of URLs from the same domain?

I have successfully figured out how to scrape statistical baseball data for a URL using BeautifulSoup. Example: I can scrape from https://www.baseball-reference.com/players/p/puckeki01.shtml for statistics for baseball player Kirby Puckett. However,…
TJE
  • 570
  • 1
  • 5
  • 20
1
vote
2 answers

Kraken-API: input-option; Python

I've been trying to load the Kraken-api-content for hours now, but it still doesn't work. My problem is the following: In the api-description https://www.kraken.com/help/api#public-market-data there it says something like "input". I.e. for the URL…
Studentu
  • 1,375
  • 2
  • 10
  • 11
1
vote
1 answer

Keep attempting urlopen until a connection is established - Python 2

I'm having a problem with my program freezing, I think it is due to no connection to the Poloniex server. How do I keep looping the urlopen request until a connection is established? Here is what I have: elif(command == "returnOrderBook"): try: …
mattstack
  • 51
  • 1
  • 12
1
vote
1 answer

Python3, Urllib.request, urlopen() is timing out

I'm using urlopen() to open a website and pull (financial) data from it. Here is my line: sourceCode = urlopen('xxxxxxxx').read() After this, I then pull the data I need out. I loop through different pages on the same domain to pull data (stock…
Micah Cobb
  • 101
  • 2
  • 9
1
vote
1 answer

Why can't I open a URL in Python 3?

I am unable to open a URL in Python3. I am using an online editor named Repl.it for coding (https://repl.it/languages/python3) Code: import urllib.request fp = urllib.request.urlopen("http://www.python.org") mybytes =…
Nirmalya Roy
  • 11
  • 1
1
vote
2 answers

urllib.request.urlopen: ValueError: unknown url type

I have a long-standing issue with urllib.request. What I do: wahlrecht = urllib.parse.quote("http://www.wahlrecht.de/umfragen/") page = urllib.request.urlopen(url) Here's the full traceback I get: Traceback (most recent call last): File…
ben0it8
  • 505
  • 1
  • 6
  • 10
1
vote
1 answer

http.client.RemoteDisconnected error while reading/parsing a list of URL's

I am working on a simple url parser: the idea is to take a url in one column, attempt to resolve it and print out the output of where it redirects to. I have the basic functionality working, however every so often it throws a…
Thomas E
  • 359
  • 1
  • 4
  • 13
1
vote
0 answers

Why is my thread getting blocked on urlopen?

while 1: try: print "It is getting blocked here" jsonResponse=urlopen(request).read() print "It is not reaching here" break except: tries+=1 print "retry after 10 seconds",tries …
anmolp95
  • 11
  • 2
1
vote
1 answer

Which registry / environment (or other) settings does urllib2.urlopen use to determine proxy settings?

Following on from this question: What is causing urllib2.urlopen() to connect via proxy? I'd like to know which operating system & environmental settings can affect urllib2.urlopen? For example, I've noticed that on some machines within our very…
Salim Fadhley
  • 22,020
  • 23
  • 75
  • 102
1 2 3
…
24 25