Stack Overflow
Stack Overflow

Questions tagged [urlopen]

The urlopen is a method of the urllib library in Python, used to open a particular URL.

The urlopen is a method of the urllib library in Python, used to open a particular URL. As a result, a file-like object is returned that contains information about the URL - headers, response data and other details about the requested URL resource.

369 questions
2
votes
2 answers

How does urllib.urlopen() work?

Let's consider a big file (~100MB). Let's consider that the file is line-based (a text file, with relatively short line ~80 chars). If I use built-in open()/file() the file will be loaded in lazy manner. I.E. if a I do aFile.readline() only a chunk…
sumid
  • 1,871
  • 2
  • 25
  • 37
2
votes
1 answer

Struggling to grab data from website using python

I'm trying to grab snowfall data from the National Weather Service at this site: https://www.nohrsc.noaa.gov/snowfall/ The data can be downloaded via the webpage with a 'click' on the file type in the drop down, but I can't seem to figure out how to…
nluchett
  • 35
  • 2
2
votes
1 answer

If I use urlretrive and urlopen to access the same url, I end up with different files. Why?

I'm fairly new to python, (and programming in general), and I've ran into some trouble while writing a program to fetch midi files from the internet. Below is some code that I expected to write two identical files: #method one url =…
user830904
  • 21
  • 1
2
votes
1 answer

Web scraping using python: urlopen returns HTTP Error 403: Forbidden

I'm trying to download data from Fragantica.com using urlopen but an error occurs ("HTTP Error 403: Forbidden") even after changing the user-agent and adding headers. I have tried the code from here as well with no success…
Noura
  • 149
  • 1
  • 2
  • 9
2
votes
0 answers

Certificate validation failed for TensorFlow hub, Elmo module

I am trying to import the Elmo module using the url: elmo = hub.Module("https://tfhub.dev/google/elmo/3", trainable=True) but am unable to load it as it is giving an error: URLError: urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate…
JBSKR
  • 33
  • 5
2
votes
1 answer

How use openurl & multiprocessing get URL different data in the same time?

This loop help get some text data from web service: while True: u = urllib2.urlopen('http://url/%d.csv' %inputd) f=open('/folder/%s.csv' $inputd,'a') csvread = csv.reader(u) csvwrite = csv.writer(f) csvwrite.writerows(csvread) …
JYCC
  • 23
  • 3
2
votes
0 answers

Urllib request takes too long to respond

Below is my code. The last line takes too long to respond. It has been more 30 minutes but no respond yet. I appreciate if anyone has any idea. import urllib.request html = urllib.request.urlopen('https://www.sahibinden.com').read()
Watzinki
  • 109
  • 1
  • 8
2
votes
1 answer

RE works in pythex but doesn't work in python

I am doing an assignment where I need to scrape information from live sites. For this I am using https://www.nintendo.com/games/nintendo-switch-bestsellers, and need to scrape the game titles, prices and then the image sources. I have the titles…
Romi Gruber
  • 23
  • 2
2
votes
1 answer

Persian characters in url and working with python urlopen() method

I need help for encoding/decoding non-ascii url to appropriate form for feeding urlopen() method. My code for scraping url(non-ascii url) from a page and going to next page: from urllib.request import urlopen from bs4 import BeautifulSoup Enterance…
Homayoun Soleimani
  • 27
  • 4
2
votes
1 answer

urlopen not working for specific url

Hello everyone I am pretty new to programming so please forgive me any noob mistakes. I am trying to use urllib in python 3.6 to find a string on a webpage and the solution I found here is working great for most webpages i tested, for example: from…
moequasar
  • 23
  • 3
2
votes
1 answer

How do I access the original response headers that contain a redirect when using urllib2.urlopen

I'm trying to parse the location header of an HTTP response that is returned after using urllib2.urlopen, but the only response headers that I receive are from the target redirect --- not the original response that contains the location header. I…
Raj
  • 3,791
  • 5
  • 43
  • 56
2
votes
1 answer

How to import ics files from a url using python

I am working on a project and this api http://icspy.readthedocs.io/en/v0.3.1/ seems good to implement the idea that I have however I face some issues and failed to find a solution and I hope this is the place to ask about this. So what I am trying…
Han
  • 131
  • 1
  • 16
2
votes
0 answers

What parameters does BeautfilSoup() accept to create BeautifulSoup object?

How am I supposed to know BeautifulSoup() accepts httpResponse object, which urlopen() returns when BeautifulSoup documentation does not mention that it does? Can someone elaborate the range of parameter types that BeautifulSoup() accepts? from bs4…
user3562812
  • 1,751
  • 5
  • 20
  • 30
2
votes
1 answer

Python 3.6.3 urlopen removing server name from URI for html file stored on remote server

I need to parse hundreds of HTML files that are archived on a server. The files are accessed via UNC, and then I use pathlib's as_uri() method to convert the UNC path to as URI. Full UNC path for example below:…
Lava Viperidae
  • 31
  • 5
2
votes
1 answer

"can't concat bytes to str" on encoded urlopen request

I am trying to make a hashed user request function using coinmate API, which returns users current balance: def getBalances(self): from urllib.request import Request, urlopen url = 'https://coinmate.io/api/balances' signature…
8ToThePowerOfMol
  • 91
  • 11
1 2 3
…
24 25