Stack Overflow
Stack Overflow

Questions tagged [urlopen]

The urlopen is a method of the urllib library in Python, used to open a particular URL.

The urlopen is a method of the urllib library in Python, used to open a particular URL. As a result, a file-like object is returned that contains information about the URL - headers, response data and other details about the requested URL resource.

369 questions
0
votes
3 answers

data from url open in list

I want to download a file from the net, i.e.: http://www.malwaredomainlist.com/hostslist/ip.txt and put that in a list to further manipulate the items in the list. I tried print "Downloading with urllib2" f = urllib2.urlopen(malwareurl) …
f0rd42
  • 1,429
  • 4
  • 19
  • 30
0
votes
1 answer

ElementTree - ParseError: not well-formed (invalid token)

I'm trying to open an XML file using urlopen and reading it. However, I keep getting the following error: xml.etree.ElementTree.ParseError: not well-formed (invalid token) Here is the code: def wx(icao): if re.search(r'!wx *\w', icao): …
Savvis
  • 53
  • 1
  • 3
  • 7
0
votes
2 answers

Improve URL reachable check

I'm currently running a python script against multiple web server. The general task is to find out broken (external) links within a cms. Script runs pretty well so far but in reason I test around 50 internal projects and each with several hundreds…
Andreas
  • 105
  • 8
0
votes
1 answer

urlopen() throwing error in python 3.3

from urllib.request import urlopen def ShowResponse(param): uri = str("mysite.com/?param="+param+"&submit=submit") print(urlopen(uri).read()) file = open("myfile.txt","r") if file.mode == "r": filelines = file.readlines() for line…
Rummy Khan
  • 69
  • 9
0
votes
0 answers

Errors with urlopen()

Good day! Wrote the following code in python 3 *, which gives a bunch of errors: from html.parser import HTMLParser import re from urllib.request import urlopen import…
Nawy
  • 13
  • 1
  • 5
0
votes
2 answers

Receiving "KeyError" after decoding json result from url

I am new to Python. I am trying to parse the json result from a url. Originally, I was using the following: response = urllib.request.urlopen(url) json_obj = json.load(response) This gave an error along the lines of "JSON object should be 'str' not…
JOEMan90
  • 5
  • 1
  • 2
0
votes
0 answers

Python's urllib2 won't connect... to anything

I can ping and can also see google.com in a browser from the same machine, but when I try to use urllib2.urlopen(url) it fails. Why? tmac:~ torobinson$ ping google.com PING google.com (4.35.2.172): 56 data bytes 64 bytes from 4.35.2.172: icmp_seq=0…
Tony
  • 49
  • 3
0
votes
3 answers

Is it possible to "refresh" a connection created with urllib2.urlopen?

I am fetching data from a URL using urllib2.urlopen: from urllib2 import urlopen ... conn = urlopen(url) data = conn.read() conn.close() Suppose the data did not "come out" as I had expected. What would be the best method for me to read it again? I…
barak manos
  • 29,648
  • 10
  • 62
  • 114
0
votes
1 answer

Python: save a page with a lot of graphics as a .html file

I want to save a visited page on disk as a file. I am using a urllib and URLOpener. I choose a site http://emma-watson.net/. The file is saved correctly as .html, but when I open the file I noticed that the main picture on top which contains…
Rop
  • 217
  • 2
  • 12
0
votes
1 answer

Urllib2 ValueError: unknown url type PNG

y is the url http://statseeker/graphs/ping.jc-4050-1.delay.1405951106.png. This is a internal website. When trying to save this PNG file this is the error it throws. I have not been able to find any info of even where to start. I want to be able to…
BilliAm
  • 590
  • 2
  • 6
  • 26
0
votes
1 answer

Handling Authentication Retrieving a Image from authenicated website Via urllib2

I am trying to make a small API that logs into a internal monitoring tool via web and retrieves images on pages that I specify using the login credentials I specify. It's not passing any authentication to the last section after it has already built…
BilliAm
  • 590
  • 2
  • 6
  • 26
0
votes
0 answers

Cache data after urllib.urlopen

I was trying to write unittest for my script which scrapes data after urllib.urlopen(url) But I want to cache the webpage itself so that the tests do not have to load the webpage every time. Is there a way to cache the data and then scrape data .I…
Ayush
  • 167
  • 3
  • 10
0
votes
1 answer

urllib ignore authentication requests

I'm having little trouble creating a script working with URLs. I'm using urllib.urlopen() to get content of desired URL. But some of these URLs requires authentication. And urlopen prompts me to type in my username and then password. What I need is…
j3nc3k
  • 65
  • 8
0
votes
0 answers

Not able to read data from url in Python

#!/usr/bin/python from urllib import urlopen import re webpage = urlopen('http://en.wikipedia.org/wiki/Python_(programming_language)').read() patFinderTitle = re.compile('(.*)') patFinderLink = re.compile('
Bad_Coder
  • 1,019
  • 1
  • 20
  • 38
0
votes
2 answers

Fail to submit webform with urlopen

I'm a total newbie on scraping but I have started on a small project using Python 3.4 For some reason the following code does not submit properly. In my first attempt I basically only want to hit "searh"("Sök") on a webform. The code I have used…
Equilib
  • 1
1 2 3
…
24 25