1

Using Rails 3.2, carrier wave, and recently switched to store on Amazon S3. My setup and uploads are all working fine. 1. I have image_uploader.rb to upload and store images. Displaying them all works fine 2. I have file_uploader.rb to upload and store files. I've even taken it a step further to upload ZIP files and extract a version so that both the ZIP file and TXT files are stored in the correct place on S3. My problem is I run a method on the TXT file. In the past, I used storage :file With that I was able to:

Dir.chdir("public/uploads/")
import_file = Dir['*.TXT'].first
f = File.new(import_file)

Now, that I'm using storage :fog I can't get seem to retrieve/File.new/Open the file. I see the file with the usual commands:

@upload1.team_file                      # stored file
@upload1.team_file.url                  # url
@upload1.team_file_url(:data_file).to_s # version created

I've been pouring through all kinds of very limited leads on retrieving and/or opening the file, but everything I try seems to return errors, such as: 

Errno::ENOENT: No such file or directory - https://teamfiles.s3.amazonaws.com/data_files…

Thoughts on the difference here of retrieving and USING a file from AmazonS3? Thanks!

Bart
  • 41
  • 4
  • get this value and try putting in a browser - `@upload1.team_file.url` , it is possible you are getting an access error? – house9 Jan 23 '14 at 21:28
  • Nice lead house9. As I mentioned, I upload the ZIP file and then extract the text file with uniq ext .abc But, from browser I noticed the errors for the MAIN file and the download for the VERSION file. From S3 Console Metadata Key: content type, Value: application/ZIP The Value needs to be text/abc which is unique extension. Once I set that, my method could open the file with the following: f = open(self.team_file_url(:data_file)) Now the trick is to figure out how to adjust the Metadata properly on the extracted file, without having to manually set it every time!!?? – Bart Jan 24 '14 at 00:00

1 Answers1

3

Pulling from multiple threads, APIs, etc. I'm answering my own question with what I've found. I welcome any corrections or improvements:

To retrieve carrierwave files uploaded to AmazonS3, you have to understand that open(@upload.file_url) or File.open(@upload.file_url) does NOT open the file, it only opens the PATH to the file. (ref: Ruby OpenURI ) I use: open_uri_url = open(@upload.file_url)

You then have to find the specific file in that path that you want. For me, I then find a ZIP file that was uploaded to AmazonS3 and Extract the specific file within the ZIP file that I want with a unique *.ABC extension:

zip_content_file = Zip::File.open(open_uri_url).map{|content| content if content.to_s.split('.').last == "ABC"}.compact.first

Now, from here, where to extract to?? I create a unique directory in the Rails tmp directory to extract the file to, use it and then delete the directory:

tmp_directory = "tmp/extracts/#{@upload.parent_id}/"
FileUtils.mkdir_p(tmp_directory) unless File.directory?(tmp_directory)
extract = zip_content_file.extract(tmp_directory + content_file.to_s)

Now with found from the AmazonS3 stored ZIP file and extracted, I can open, read, etc:

f = File.new(tmp_directory + extract.to_s)

I hope this helps with Carrierwave, AmazonS3, ZIP files and using them once uploaded.

Bart
  • 41
  • 4