Storing image using open URI and paperclip having size less than 10kb

Question

I want to import some icons from my old site. The size of those icons is less than 10kb. So when I am trying to import the icons its returning stringio.txt file.

require "open-uri"
class Category < ActiveRecord::Base
   has_attached_file :icon,  :path => ":rails_root/public/:attachment/:id/:style/:basename.:extension"
  def icon_from_url(url)
    self.icon = open(url)
   end    
end

In rake task.

   category = Category.new
   category.icon_from_url "https://xyz.com/images/dog.png"
   category.save

You're trying to retrieve images from another site, having some sort of problem, yes? Why the paperclip stuff then? Why don't you just download the icons and upload them separately? You do realize that paperclip if for handling file uploads, right? — coreyward, Jun 12 '11 at 00:17
i want to add the icons in paperclip style. There are some 400 icons..its not possible for me to create tons of folders. — Mohit Jain, Jun 12 '11 at 06:33
@coreyward more over these icons are related to some category and each category has lots of other details. — Mohit Jain, Jun 12 '11 at 19:52
@Michaël-Witrant The size of those icons is less than 10kb. So when I am trying to import the icons its returning stringio.txt file. — Mohit Jain, Jun 13 '11 at 07:12
So you want to change the name of the files paperclip saves? — Michaël Witrant, Jun 13 '11 at 07:24
no. I want to save those files. I can downlaod 300 icons and place it in directories like paperclip does. But when i am trying to fecth those files its returning stringio.txt files as their sizes are less than 10kb. — Mohit Jain, Jun 13 '11 at 08:08
What does the file contain? I tried that on a Paperclip attachment: the filename is stringio.txt but it contains the image. — Michaël Witrant, Jun 13 '11 at 19:41

score 36 · Accepted Answer · answered Jun 12 '11 at 22:12

36

Try:

def icon_from_url(url)
  extname = File.extname(url)
  basename = File.basename(url, extname)

  file = Tempfile.new([basename, extname])
  file.binmode

  open(URI.parse(url)) do |data|  
    file.write data.read
  end

  file.rewind

  self.icon = file
end

answered Jun 12 '11 at 22:12

Kevin Sylvestre

37,288
33
152
232

this code worked for the first image but then nothing. I have a loop of some 350 images. I double checked the links. its perfectly fine. In addition image name is not the same. – Mohit Jain Jun 13 '11 at 06:10
@Mohit Are you calling `save` on your model after performing this method? – Kevin Sylvestre Jun 13 '11 at 16:26
3

Setting `original_filename` on the IO returned by open-uri is much simpler and faster. – Michaël Witrant Jun 18 '11 at 08:37
@MichaëlWitrant I found that PaperClip 3.5.1 performs MIME type validations based on the Tempfile's filename. It doesn't matter if `original_filename` is set. I used @KevinSylvestre's method to force the Tempfile to have the correct extension, and thus PaperClip "guessed" the correct MIME type. (You could argue that the validation is useless in this scenario, but having the correct MIME type stored in the DB might be useful.) – mpoisot Oct 07 '13 at 19:45

score 9 · Answer 2 · answered Jun 13 '11 at 07:48

9

To override the default filename of a "fake file upload" in Paperclip (stringio.txt on small files or an almost random temporary name on larger files) you have 2 main possibilities:

Define an original_filename on the IO:

def icon_from_url(url)
  io = open(url)
  io.original_filename = "foo.png"
  self.icon = io
end

You can also get the filename from the URI:

io.original_filename = File.basename(URI.parse(url).path)

Or replace :basename in your :path:

has_attached_file :icon, :path => ":rails_root/public/:attachment/:id/:style/foo.png", :url => "/:attachment/:id/:style/foo.png"

Remember to alway change the :url when you change the :path, otherwise the icon.url method will be wrong.

You can also define you own custom interpolations (e.g. :rails_root/public/:whatever).

answered Jun 13 '11 at 07:48

Michaël Witrant

7,525
40
44

3

I am trying to use this but if I try to set original_filename on io in this example I am getting an NoMethodError: undefined method `original_filename=' Error on the tempfile. Ruby 1.8.7 rails 2.3.8 – Dustin M. May 16 '12 at 21:16
4

@DustinM. This works: `def io.original_filename; "foo.png"; end` – Henrik N Sep 27 '12 at 17:20
I can confirm that this works. In order to access a local variable, I had to use define_method: open(doc.url) do |file| file.singleton_class.instance_eval do define_method(:original_filename) { doc.title } end end – Paul Brannan Nov 09 '12 at 20:19
This doesn't work due to the undefined method error. – Joshua Plicque Apr 20 '16 at 15:11

score 1 · Answer 3 · edited May 23 '17 at 12:07

1

You can also disable OpenURI from ever creating a StringIO object, and force it to create a temp file instead. See this SO answer:

Why does Ruby open-uri's open return a StringIO in my unit test, but a FileIO in my controller?

edited May 23 '17 at 12:07

Community

1
1

answered Mar 22 '13 at 19:37

Micah Winkelspecht

1,805
1
16
7

score 1 · Answer 4 · answered Jun 12 '11 at 21:55

1

You are almost there I think, try opening parsed uri, not the string.

require "open-uri"
class Category < ActiveRecord::Base
   has_attached_file :icon,  :path =>:rails_root/public/:attachment/:id/:style/:basename.:extension"
  def icon_from_url(url)
    self.icon = open(URI.parse(url))
  end    
end

Of course this doesn't handle errors

answered Jun 12 '11 at 21:55

Tadas T

2,492
20
20

Its not going to save extenstion! – Nadeem Yasin Oct 05 '13 at 21:36

score -2 · Answer 5 · answered Jun 15 '11 at 02:25

In the past, I found the most reliable way to retrieve remote files was by using the command line tool "wget". The following code is mostly copied straight from an existing production (Rails 2.x) app with a few tweaks to fit with your code examples:

class CategoryIconImporter
  def self.download_to_tempfile (url)
    system(wget_download_command_for(url))
    @@tempfile.path
  end

  def self.clear_tempfile
    @@tempfile.delete if @@tempfile && @@tempfile.path && File.exist?(@@tempfile.path)
    @@tempfile = nil
  end

  def self.set_wget
    # used for retrieval in NrlImage (and in future from other sies?)
    if !@@wget
      stdin, stdout, stderr = Open3.popen3('which wget')
      @@wget = stdout.gets
      @@wget ||= '/usr/local/bin/wget'
      @@wget.strip!
    end
  end
  def self.wget_download_command_for (url)
    set_wget
    @@tempfile = Tempfile.new url.sub(/\?.+$/, '').split(/[\/\\]/).last
    command = [ @@wget ]
    command << '-q'
    if url =~ /^https/
      command << '--secure-protocol=auto'
      command << '--no-check-certificate'
    end
    command << '-O'
    command << @@tempfile.path
    command << url
    command.join(' ')
  end

  def self.import_from_url (category_params, url)
    clear_tempfile

    filename = url.sub(/\?.+$/, '').split(/[\/\\]/).last
    found = MIME::Types.type_for(filename)
    content_type = !found.empty? ? found.first.content_type : nil

    download_to_tempfile url

    nicer_path = RAILS_ROOT + '/tmp/' + filename
    File.copy @@tempfile.path, nicer_path

    Category.create(category_params.merge({:icon => ActionController::TestUploadedFile.new(nicer_path, content_type, true)}))
  end
end

The rake task logic might look like:

[
  ['Cat', 'cat'],
  ['Dog', 'dog'],
].each do |name, icon|
  CategoryIconImporter.import_from_url {:name => name}, "https://xyz.com/images/#{icon}.png"
end

This uses the mime-types gem for content type discovery:

gem 'mime-types', :require => 'mime/types'

Storing image using open URI and paperclip having size less than 10kb

5 Answers5

Linked