1

I want to use Amazon S3 to store all of my files for my websites. I can set S3QL up properly so that it is mounted on /var/www and everything works fine.

I want to do it this way so that I can create an AMI, and then fire up multiple instances which would then also mount the same S3 bucket to /var/www

Will this cause any data corruption issues? Is this good practice?

EDIT: This would be for read+write

ZeunO8
  • 113
  • 1
  • 6
  • You might need to read this - http://serverfault.com/questions/353808/is-mounting-s3-buckets-directly-into-ec2-instances-safe – Daniel t. Feb 14 '13 at 21:55

1 Answers1

1

If: you mount the bucket read only, you're fine.

If: you try to mount it read/write, you're headed for trouble.

A method I've used is to create a large file, roughly the size of a disk that would hold the data and treat it like a partition. Essentially, it becomes a large file container. This saves a huge amount of time for things like copying small files, since you're not traversing actual individual files and directories over the network.

Stephan
  • 999
  • 7
  • 11
  • What? A filesystem on an image file in S3?? – mgorven Feb 14 '13 at 21:53
  • Oh, S3QL chunks files. That's slightly less crazy then. – mgorven Feb 14 '13 at 21:55
  • I wasn't speaking specifically about S3QL, just offering a general comment. Think of it like mounting an ISO remotely; instead of parsing through dozens of directories and initiating file transfers individually, it just reads the raw block data based on it's offset location. Works great with nfs. – Stephan Feb 14 '13 at 21:57
  • For reading, okay, but writing will be horrible because S3 doesn't support random writes. – mgorven Feb 14 '13 at 21:59
  • But why are you headed for trouble with read/write? – ZeunO8 Feb 14 '13 at 22:09
  • Imagine you have server A and B both mounted S3 bucket SHARE. A and B both decide to edit FILE.txt. Both save different data at the same time. Which server is right? – Stephan Feb 14 '13 at 23:58
  • @Twitchy [mutex](http://en.wikipedia.org/wiki/Mutual_exclusion) – New Alexandria Mar 23 '13 at 22:44
  • @Stephan I was going through S3QL docs and they are strongly against multiple mounts. Though they dont specify for read-only mounts. I want to have one read/write and multiple read mounts at the same time being accessed. Will there be any issues or everything will be fine? – Zia Ul Rehman Mughal Sep 22 '17 at 06:23
  • 1
    @ZiaUlRehmanMughal So long as the volume is mounted read-only, there should be no issues. If there's one read/write mount, you do run the risk of writing the data that another node is reading, causing inconsistencies. If you think that is an issue, you might be better off using EFS (or NFS, Ceph, Gluster, or another, more sophisticated solution.) – Stephan Sep 29 '17 at 04:51