Server Fault Stack Exchange
Server Fault Stack Exchange

Questions tagged [amazon-s3]

This is for questions on Amazon's cloud storage service.

1120 questions
5
votes
2 answers

Snapshots == EBS backed instance, persistence, and S3 storage?

I want to clear up some confusion and clarify understanding on EC2 instances being EBS backed. If you start an AMI from a community image, configure your server as you want it configured, then use the "create image" option from the EC2 dashboard to…
Bart Silverstrim
  • 31,172
  • 9
  • 67
  • 87
5
votes
2 answers

What design features make Joyent's ZFS and Amazon's EBS (S3) reliable?

I know this isn't exactly an apples to apples comparison, what I'm trying to evaluate is which one is safer - less likely to lose data. Joyent's SmartOS uses ZFS to store data whereas an EC2 machine can use Amazon Elastic Block Store (EBS) which…
Akshat
  • 435
  • 5
  • 11
5
votes
1 answer

Using an IP Address to Point to Amazon S3 Storage

Is there a way to get an IP address to refer to Amazon S3 Storage location? The current solution my company is thinking of going with involves setting up a EC2 server with an Elastic IP and using Squid as a Proxy to the S3 location. Is there a…
Tyndall
  • 591
  • 1
  • 7
  • 18
5
votes
2 answers

S3 based file system capable of requesting only part of file

I'm storing large datasets in s3, but on a given computer in my cluster, my program only needs to read a small subset of the data. I first tried s3fs, but it downlooads the entire file first, which takes a really long time. Are there any s3 backed…
UsAaR33
  • 1,096
  • 3
  • 11
  • 20
5
votes
2 answers

Webstats for large Amazon S3/Cloudfront logs?

I'm trying to find a way to parse our Amazon S3 access logs to get some webstats. I've been trying to use AWStats 7, but I got to the point of where after day 9 of a given month it can't process any more logs because it runs out of memory. This…
Mxx
  • 2,362
  • 2
  • 28
  • 40
5
votes
1 answer

Amazon S3 Set Expires header

Is there any way to tell Amazon S3 that all existing and future files in some bucket should have some speicific Expires header?
SiberianGuy
  • 635
  • 1
  • 8
  • 19
5
votes
2 answers

Is there software that can help me easily back up my data from Amazon S3 to my premise?

So everyone backs their stuff up to the cloud. But I want to back up my stuff from the cloud. For a planned SaaS service, users are saving files containing their intellectual property and we're saving them in S3. I'm not entirely comfortable with…
Ernest Mueller
  • 1,199
  • 2
  • 12
  • 25
5
votes
5 answers

What sort of web server hardware do you use to handle 100 Mbps+ of static files?

I currently use Amazon S3 for much of my static file serving needs but my monthly bill is getting very expensive. I did some rough calculations using the logs and at peak times, my most expensive Amazon bucket is handling about 100 180 Mbps of…
outcassed
  • 1,290
  • 1
  • 9
  • 11
5
votes
2 answers

CNAME record for Amazon S3 -- any drawbacks?

I'd like to make a CNAME record for my Amazon S3 domain, to have nicer URLs. There's not much discussion of this out there on the web, and nothing at all in the Amazon documentation (unless I'm missing it). Is this a reasonable thing to do? Are…
John Bachir
  • 2,364
  • 7
  • 29
  • 37
4
votes
0 answers

Understand S3 cost at folder level

I am planning for a use case wherein which my S3 bucket is used by 10 different users. All these users has separate folders within this bucket, to where they'll store their files. Now I want to know how much storage (and there by the cost) does each…
serverstackqns
  • 764
  • 3
  • 16
  • 42
4
votes
0 answers

How to remove orphaned s3 bucket

I have an s3 bucket that I can see in the console or with aws s3 ls/aws s3api list-buckets but the bucket appears to be in some sort of orphaned state. It's missing the Access and Region fields in the s3 console and I can't access anything about the…
Robert
  • 141
  • 2
4
votes
4 answers

How can I get a breakdown of s3 cost at per bucket level in Cost Explorer?

It seems like I can get total cost of S3 using the service level in the filter. However I want to get cost per bucket level. Can it be done in Cost Explorer? If not, can I get the breakdown with aws cli?
Anthony Kong
  • 3,288
  • 11
  • 57
  • 96
4
votes
1 answer

How to make objects uploaded to S3 bucket readable without explicitly setting their permissions

I want to make objects uploaded to S3 bucket readable without explicitly setting their permissions. Relevant discussion I found about Digital Ocean S3 is here. As in the above and other discussions, I was able to find solutions to set permissions…
PatrizioRD
  • 41
  • 1
4
votes
2 answers

Restrict access to S3 bucket folders to specific website users? (not using IAM Users)

I have a website where users need to log in. They can upload and delete their own pictures BUT these pictures are supposed to be private so images are not set to public that anyone can view. I know that we can use IAM policies to restrict access to…
Dora
  • 341
  • 1
  • 5
  • 15
4
votes
2 answers

How to create an OFFLINE Incremental backup of an AWS S3 bucket

I'm looking for a way to take daily backups of an AWS bucket as incremental backups. These are to be stored offline and away from AWS. For other storage systems (such as NAS drives) I use a daily rsync for backups. Using rsync's --link-dest…
Philip Couling
  • 1,682
  • 1
  • 19
  • 37
1 2 3
…
74 75