1

We are trying to use the d2.xlarge instance type for worker nodes in Hadoop cluster.

When i look at the AWS Site https://aws.amazon.com/ec2/instance-types/

It says HDD based local storage.

Is it EBS Storage? or attached to the EC2 instance?

Shankar
  • 8,529
  • 26
  • 90
  • 159

2 Answers2

1

Local means attached storage. HDD means spinning disks (not SSD's).

Evert
  • 93,428
  • 18
  • 118
  • 189
1

You can attach Elastic Block Store (EBS) volumes to any Amazon EC2 instance. EBS is persistent disk storage.

Some EC2 instances also have Instance Store, which is locally-attached disk storage. If the instance is stopped or terminated, the contents of Instance Store is lost.

Instance Store is popular for use with Amazon EMR because it provides very large amounts of storage for HDFS. However, please be aware that the data is lost if the cluster is terminated.

The d2.xlarge instance type has 3 x 2000 GB instance store drives, which are stored on magnetic disk. This is in addition to any EBS volumes you attach.

John Rotenstein
  • 241,921
  • 22
  • 380
  • 470
  • Is EBS distributed File System? or is it same like S3 Object storage? – Shankar Mar 30 '18 at 08:54
  • EBS is a virtual disk. It can only be attached to one EC2 instance at a time, so it is not a distributed file system. However, you mentioned that you are running Hadoop. Hadoop will create the Hadoop Distributed File System (HDFS) on top of all available disks in the cluster. – John Rotenstein Mar 30 '18 at 10:34