OpenURI is a Ruby module included in the standard library, which provides an easy-to-use wrapper for net/http, net/https and net/ftp.
Questions tagged [open-uri]
311 questions
18
votes
5 answers
Rails 3.2.17 Runtime Error Redirection Forbidden facebook
I have this code I use to get avatars from Facebook...
if auth.info.image.present?
user.update_attribute(:avatar, URI.parse(auth.info.image))
end
When I try to load the code now I get this error:
A RuntimeError occurred in…
mystic cola
- 1,465
- 1
- 21
- 39
16
votes
8 answers
How should my scraping "stack" handle 404 errors?
I have a rake task that is responsible for doing batch processing on millions of URLs. Because this process takes so long I sometimes find that URLs I'm trying to process are no longer valid -- 404s, site's down, whatever.
When I initially wrote…
Mario Zigliotto
- 8,315
- 7
- 52
- 71
14
votes
6 answers
Ruby open-uri open method loses file extension opening images
I'm using ruby 1.9.2 along with Rails 3.1.4 and Paperclip 2.4.5.
My issue is trying to save a paperclip attachment from a URI loses the file extension and saves the file without one resulting in issues with things like fancybox that require an…
Jimmy
- 9,686
- 14
- 59
- 78
13
votes
2 answers
404 error with open-uri in a rake task... what's causing it?
I have a rake task that fetches JSON data from an API, parses it, and saves it to the database:
task :embedly => :environment do
require 'json'
require 'uri'
require 'open-uri'
Video.all.each do |video|
json_stream =…
Justin Meltzer
- 13,318
- 32
- 117
- 182
12
votes
3 answers
`write': "\xCF" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError) while writing to file from url
I am getting error:
write': "\xCF" from ASCII-8BIT to UTF-8 (Encoding::UndefinedConversionError)
from line:
open(uri) {|url_file| tempfile.write(url_file.read)}
relevant code is:
require 'tempfile'
require 'open-uri'
require 'uri'
..
uri =…
Vadim
- 633
- 1
- 8
- 17
12
votes
1 answer
open-uri is not redirecing http to https
I am using Hpricot and OpenURI to parse webpages and extract URLs from them.
When I get a link like "http:rapidshare.com", it is not redirecting to https.
This is the error I…
leonidus
- 363
- 1
- 3
- 11
11
votes
5 answers
How to Process Items in an Array in Parallel using Ruby (and open-uri)
I am wondering how i can go about opening multiple concurrent connections using open-uri? i THINK I need to use threading or fibers some how but i'm not sure.
Example code:
def get_doc(url)
begin
Nokogiri::HTML(open(url).read)
rescue…
Mario Zigliotto
- 8,315
- 7
- 52
- 71
10
votes
2 answers
Rails rake task fails just in production: "NoMethodError: private method `open' called for URI:Module"
I am coding a rake task which function is getting info from other webpage. To do that I use open-uri and nokogiri. I have tested in development and it does the job, but then I deploy to the production server and fails.
This is the code:
require…
Juanse Cora
- 755
- 9
- 19
9
votes
2 answers
How do I add encoded query values to a URL?
I am looking for a convenient and functional way to add encoded values to a URL query string in Ruby.
Currently, I have:
require 'open-uri'
u = URI::HTTP.new("http", nil, "mydomain.example", nil, nil, "/tv", nil, "show=" + URI::encode("Rosie &…
SimonMayer
- 4,719
- 4
- 33
- 45
9
votes
2 answers
how to test open-uri url exist before processing any data
I'm trying to process content from a list of links using "open-uri" in ruby (1.8.6), but the bad thing happens when I'm getting an error when one link is broken or requires authentication:
open-uri.rb:277:in `open_http': 404 Not Found…
Kostas
- 135
- 1
- 7
9
votes
3 answers
Copy/paste phone number into keypad or dial a toll free number in iOS programatically
I am trying to make a call from my app using
[[UIApplication sharedApplication] openURL:[NSURL URLWithString:@"telprompt://1800000002"]];
This is a toll free number in India.But while dialing, it is converting to +1(800)-000-000 (Converting and…
soumya
- 3,801
- 9
- 35
- 69
9
votes
3 answers
How to get HTTP headers before downloading with Ruby's OpenUri
I am currently using OpenURI to download a file in Ruby. Unfortunately, it seems impossible to get the HTTP headers without downloading the full file:
open(base_url,
:content_length_proc => lambda {|t|
if t && 0 < t
pbar =…
ePirat
- 1,068
- 11
- 20
9
votes
2 answers
Convert latin1 string to utf8?
how can I convert a string, that contains latin1 characters to utf8?
The string is a document, that is opened by open-uri and that contains these special characters.
Best regards
brainfck
- 9,286
- 7
- 28
- 29
9
votes
1 answer
Display HTTP headers using Open::URI?
with Open::URI, I can do the following:
require 'open-uri'
#check status
open('http://google.com').status
#get entire html
open('http://google.com').read
Is it possible to get the HTTP headers of a request so things can be debugged, something like…
CuriousMind
- 33,537
- 28
- 98
- 137
9
votes
1 answer
HTML is read before fully loaded using open-uri and nokogiri
I'm using open-uri and nokogiri with ruby to do some simple webcrawling.
There's one problem that sometimes html is read before it is fully loaded. In such cases, I cannot fetch any content other than the loading-icon and the nav bar.
What is the…
Chelsea White
- 285
- 5
- 19