10

How do I upload a CSV file from my local machine to my AWS S3 bucket and read that CSV file?

bucket = aws_connection.get_bucket('mybucket')
#with this i am able to create bucket
folders = bucket.list("","/")
  for folder in folders:
  print folder.name

Now I want to upload csv into my csv and read that file.

Cale Sweeney
  • 1,014
  • 1
  • 15
  • 37
TB.M
  • 363
  • 3
  • 8
  • 26

1 Answers1

20

So you're using boto2 -- I would suggest to move to boto3. Please see below some simple examples:

boto2

upload example

import boto 
from boto.s3.key import Key
bucket = aws_connection.get_bucket('mybucket')
k = Key(bucket)
k.key = 'myfile'
k.set_contents_from_filename('/tmp/hello.txt')

download example

import boto
from boto.s3.key import Key

bucket = aws_connection.get_bucket('mybucket')
k = Key(bucket)
k.key = 'myfile'
k. get_contents_to_filename('/tmp/hello.txt')

boto3

upload example

import boto3
s3 = boto3.resource('s3')
s3.Object('mybucket', 'hello.txt').put(Body=open('/tmp/hello.txt', 'rb'))

or simply

import boto3
s3 = boto3.resource('s3')
s3.meta.client.upload_file('/tmp/hello.txt', 'mybucket', 'hello.txt')

download example

import boto3
s3 = boto3.resource('s3')
s3.meta.client.download_file('mybucket', 'hello.txt', '/tmp/hello.txt')
print(open('/tmp/hello.txt').read())
Frederic Henri
  • 51,761
  • 10
  • 113
  • 139
  • And how to give permission..How to make private bucket.IF my bucket is already there.how can i make it private? – TB.M Nov 22 '16 at 09:16
  • best is to work with the bucket policy directly from the aws console – Frederic Henri Nov 22 '16 at 09:49
  • Great answer. Is there a reason you prefer ```s3 = boto3.resource('s3') s3.meta.client.download_file('mybucket', 'hello.txt', '/tmp/hello.txt')``` rather than ```s3 = boto3.client('s3') s3.upload_file( '/tmp/hello.txt', 'mybucket', 'hello.txt')``` ? – Peter Oct 11 '17 at 20:45
  • @Peter the `upload_file` method will let you pass an `ExtraArgs` parameter if needed (for example, set ACL permissions etc) – Frederic Henri Oct 12 '17 at 11:57
  • declaring `bucket` isn't necessary in your first upload example for boto3 – jspinella Nov 29 '22 at 07:12
  • @jspinella good eyes :) – Frederic Henri Nov 29 '22 at 08:54