Stack Overflow
Stack Overflow

Questions tagged [urlopen]

The urlopen is a method of the urllib library in Python, used to open a particular URL.

The urlopen is a method of the urllib library in Python, used to open a particular URL. As a result, a file-like object is returned that contains information about the URL - headers, response data and other details about the requested URL resource.

369 questions
0
votes
3 answers

Using urlopen to open list of urls

I have a python script that fetches a webpage and mirrors it. It works fine for one specific page, but I can't get it to work for more than one. I assumed I could put multiple URLs into a list and then feed that to the function, but I get this…
Eric Lagergren
  • 501
  • 2
  • 7
  • 18
0
votes
1 answer

How to avoid country-based redirects with urlopen or urllib2 in Python

I am using Python 2.7. I want to open the URL of a website and extract information out of it. The information I am looking for is within the US version of the website (http://www.thewebsite.com) . Since I am based in Canada, I get automatically…
LaGuille
  • 1,658
  • 5
  • 20
  • 37
0
votes
2 answers

Open an URL in Python

How can I open an URL with a variable and key values in it in Python? I tried as below, but I got a 400 bad request. Opening the file (host) in my web browser doesn't seem to be any problem though. Perhaps the proxy? for verb in verbs: host =…
maikovich
  • 368
  • 2
  • 3
  • 16
0
votes
1 answer

get source html in local system python

Dears I want get source page but not in internet rather in local system example : url=urllib.request.urlopen ('c://1.html') >>> import urllib.request >>> url=urllib.request.urlopen ('http://google.com') >>> page =url.read() >>>…
aliweb
  • 29
  • 1
  • 1
  • 4
0
votes
1 answer

If Statement Not Working For Scraper

I'm hoping you can show me where i'm going wrong with my webscraper. What I would like to do is to be notified when a certain string ("Sorry, Gruen Fan") changes on a page. I'm able to pull in the string, however, the "If" function doesn't seem to…
Sam Perry
  • 2,554
  • 3
  • 28
  • 29
0
votes
0 answers

Urlopen hangs with site

I'm using urlopen to read a site but it hangs in my program. But when I'm testing from terminal and python interpreter it works fine. The code is this: page = urlopen ("http://mangafox.me/directory/").read () print ('soup') # soup = BeautifulSoup…
Clepto
  • 685
  • 1
  • 6
  • 7
0
votes
1 answer

Python Urllib Urlopen won't return after new line?

I'm trying to get authenticated by an API I'm attempting to access. I'm using urllib.parse.urlencode to encode the parameters which go in my URL. I'm using urllib.request.urlopen to fetch the content. This should return 3 values from the server,…
Rob M
  • 298
  • 1
  • 4
  • 18
0
votes
1 answer

Set a timer on urlopen so when connection times out it tries to reconnect

I have a question on Python 2.7. The urlopen function sometimes fails to connect due to the bad internet. So I want to add a timer on it, so that when the connection times out, it close the previous connection and try to reconnect again. Here in my…
Robert
  • 2,189
  • 6
  • 31
  • 38
0
votes
0 answers

Python3: byte array decoding from urlopen

I'm trying to use python to find some words across webpages (just to practice) but I keep running into a problem. This is it: url = 'someWikipage' hdrs = { 'User-Agent': "Mozilla/5.0 (X11; U; Linux i686) Gecko/20071127 Firefox/2.0.0.11" } req =…
Tim
  • 2,000
  • 4
  • 27
  • 45
0
votes
3 answers

How to count the number of time a unique URL is open in python?

I'm running a Python code that reads a list of URLs and opens each one of them individually with urlopen. Some URLs are repeated in the list. An example of the list would be something like: www.example.com/page1…
LaGuille
  • 1,658
  • 5
  • 20
  • 37
0
votes
2 answers

Why does python urllib2 urlopen return something different from the browser on API call

I need to routinely access and parse XML data from a website of the form: https://api.website.com/stuff/getCurrentData?security_key=blah I cannot post the actual connections because of the secure nature of the data. When I put this url into my…
jessi
  • 1,438
  • 1
  • 23
  • 36
0
votes
3 answers

Downloading web image w/ urlopen/shutil : error __exit__

I'm trying to download images with shutil/urlopen because of deprecated? I'm not sure if its deprecation, but urlretrieve doesn't download the file, it just creates folder of the image name instead. After looking at other question I saw one that…
Brandon Nadeau
  • 3,568
  • 13
  • 42
  • 65
0
votes
1 answer

django - configure urls.py regex to process an urlopen post

I am a total newbie to Python and Django and am playing around with passing data, via http post, to a simple web app. I went through the Django tutorial and have a decent understanding of what was being taught but I don't quite have the urls.py…
user2234592
  • 1
  • 2
0
votes
2 answers

HTTP Request with Python 2.7.3

I have a string result in this string is the URL www.test.com I know that on www.test.com is an website with the number 4. I will save the number in my programm as an integer. import urllib giveTheInt = [urllib.urlopen(url)] But i receive…
user1865863
  • 29
  • 1
  • 5
0
votes
2 answers

open file with string that has multiple variables

I can get the text file when I set airport to one variable. However, how can I get the text files for multiple airport codes and display the information? airport = 'KSFO, KSJC, KOAK' for metar in…
Savvis
  • 53
  • 1
  • 3
  • 7
1 2 3
…
24 25