0

How to share S3 storage between multiple EC2 instances? I am beginner to AWS, I need to know how to share a drive between multiple EC2 instances.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
chandrakumar
  • 1
  • 2
  • Does it have to be a disk? Cannot you use an object storage system (like S3), a database or something like that? – Thilo Oct 06 '15 at 12:19
  • 3
    Possible duplicate of [Shared storage between multiple AWS EC2 instances](http://stackoverflow.com/questions/17415620/shared-storage-between-multiple-aws-ec2-instances) – Thilo Oct 06 '15 at 12:20
  • Also http://stackoverflow.com/questions/30368893/how-to-setup-shared-persistent-storage-for-multiple-aws-ec2-instances?rq=1 – Thilo Oct 06 '15 at 12:25
  • Yes, i want to use like drive, Is there any other possible way to do this? i want to use like shared drive between multiple instances? i don't want to use other third party tool. i want to make use of resources provided by AWS. – chandrakumar Oct 06 '15 at 13:10
  • 1
    Either setup a server as an NFS server or use Amazon Elastic File System if you can. But if you can avoid it you should. Look at using S3 or another method. Think about when you have 5 servers or 15 servers all trying to access the same data. It can be a pain – greg_diesel Oct 06 '15 at 15:04

2 Answers2

2

Currently you can't, and S3 is your best bet, but AWS does have their Elastic File System in BETA currently, and there is the possibility it will be available for general availability anytime (I have no inside knowledge, just a guess - maybe even this week, they often have lots of announcements during their annual conference going on now).

You can signup for 'preview' access and see if it suits your needs, and then decide if you can wait for it to become fully available.

AWS EFS will allow you to share a drive between instances:

Amazon EFS supports the Network File System version 4 (NFSv4) protocol, so the applications and tools that you use today work seamlessly with Amazon EFS. Multiple Amazon EC2 instances can access an Amazon EFS file system at the same time, providing a common data source for workloads and applications running on more than one instance.

https://aws.amazon.com/efs/

E.J. Brennan
  • 45,870
  • 7
  • 88
  • 116
0

EFS (still in beta, half a year later) indeed looks like the best option. But as EFS is basically just a managed, highly available NFS server, it should be possible to roll out some other NFS solution first, and replace it with EFS once it's finally available.

One promising candidate seems dCache, which is

a system for storing and retrieving huge amounts of data, distributed among a large number of heterogenous server nodes, under a single virtual filesystem tree with a variety of standard access methods.

It is used by research institutions all over the world to store over 100PB of data, and it provides an NFSv4 interface. Not sure how easy setup on AWS would be, or what the performance would be like.

https://www.dcache.org/

tlossen
  • 1,247
  • 1
  • 8
  • 7