Accessing data stored in Amazon S3 through Rails

Question

My goal is to make graphs from data within excel files that users have uploaded to Amazon S3.

I've already implemented the functionality for users to upload the excel files with Carrierwave, now I need to be able to access the data and make it presentable for use with a charting library (Highcharts).

The task that I am stuck on is directly accessing the data in S3 through Rails. Once the data is pulled it should be fairly straightforward to manipulate it with Highcharts.

Any suggestions would be much appreciated!

LandonSchropp · Accepted Answer · 2013-04-15T17:59:23.200

5

You can use the AWS SDK:

require 'aws-sdk'

# retrieve the access key and secret key
access_key_id = ENV["ACCESS_KEY_ID"]
secret_access_key = ENV["SECRET_ACCESS_KEY"]

# create an instance of the s3 client
s3 = AWS::S3.new(access_key_id: access_key_id, secret_access_key: secret_access_key)

# get the bucket
bucket = s3.buckets['your-bucket-name']

# retrieve the objects
bucket.objects.each do |object|
  puts object.key
  puts object.read
end

edited Apr 15 '13 at 17:59

answered Apr 14 '13 at 20:24

LandonSchropp

10,084
22
86
149

Also, would there be security implications for placing the access key info directly into the function? – Anconia Apr 14 '13 at 22:39
Sorry, I didn't post a complete implementation. You're right; you shouldn't put your keys directly in the code. I usually [stick them in an environment variable](http://stackoverflow.com/a/11300680/262125). I'll update my answer. – LandonSchropp Apr 15 '13 at 17:58
@LandonSchropp, I am getting AWS::S3::Errors::AccessDenied: Access Denied , created from aws newly key and secret added policy also , what I am doing wrong at s3 setup ? – sunil Mar 30 '21 at 11:41

Taimoor Changaiz · Answer 2 · 2015-10-30T17:10:27.253

s3 = Aws::S3::Client.new
bucket = Aws::S3::Bucket.new('AWS_BUCKET NAME HERE')
bucket.objects.each do |obj|
  File.open("#{Rails.root}/#{obj.key}", 'wb') do |file|
    s3.get_object( bucket:ENV[:AWS_BUCKET], key: obj.key , response_target: file)
  end
end

OR

s3 = Aws::S3::Client.new
s3.list_objects(bucket: 'AWS_BUCKET NAME HERE').each do |response|
  response.contents.each do |obj|
    File.open("#{Rails.root}/#{obj.key}", 'wb') do |file|
      s3.get_object( bucket: 'AWS_BUCKET NAME HERE', key: obj.key , response_target: file)
    end
  end
end

There is official AWS-SDK RUBY gem

AWS SDK ruby official documentation for version 2

For enviornment variable configuration you can figaro or dotenv (for development environment) or set in ~/.bashrc file.

Note:

You need to create S3 bucket and get AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY.
Before development you can test and access your s3 bucket data using google chrome s3 browser extenstion
Run command source ~/.bashrc or . ~/.bashrc file to reflect changes, if you store ENV variables there.

Code Reference

Accessing data stored in Amazon S3 through Rails

2 Answers2