How do I attach, format and mount a volume on boot in ecs?

Question

I want my persistence (ebs) volume to be separate from my auto-scaling group launch configuration. So that it won't be accidentally auto deleted by terraform, or something. Also I may do something like /mnt/taskname so that I have a different persistence volume for each task that needs data.

Here's how I think it would work.

install aws-cli for some reason it's missing (packages in cloud init), which happens late?
use aws command to attach the volume
format the volume using fs_setup in cloud init
mount the partitions using mounts in cloud init

Problem is that this won't work though because 1 wouldn't happen until after 3 and 4 (I think), and I'm not really sure if 2 is the right way to go to attach the volume.

Maybe I shouldn't be using the ecs optimized image? seems like that would solve my issue, but then I'd have to figure out what to change the amazon (base?) image to have it otherwise configured like the ecs image? maybe there's another way to attach the volume than the aws command? maybe some other magic than what I'm considering entirely?

You mentioned Terraform, if you are using Terraform why not attach EBS to your EC2 instance using Terraform? That way you will have volume attached fans ready to be formatted and mounted when the OS boots. — Andrey, May 09 '17 at 19:43
@Andrey there doesn't seem to be a way to attach an .. external volume (one that's not part of the launch configuration), at least not that I could figure out. and using attach volume requires an instance id, which doesn't exist until the autoscaling group starts it. If you know a way to do that... — xenoterracide, May 10 '17 at 00:18

xenoterracide · Answer 1 · 2017-05-11T15:36:26.423

This is a partial answer as i'm still working through it, first, I'm using a plain Amazon Linux AMI, not the ECS optimized one (which is out of date...)

data "template_file" "AttachVolume" {
  template = "${file("cloudinit/attach_volume.sh")}"

  vars {
    volume = "${aws_ebs_volume.Nexus.id}"
  }
}

data "template_cloudinit_config" "CloudInit" {
  part {
    filename     = "fs.cfg"
    content_type = "text/cloud-config"
    content      = "${file("cloudinit/fs.yml")}"
  }
  part {
    filename = "attach_volume.sh"
    content_type = "text/cloud-boothook"
    content = "${data.template_file.AttachVolume.rendered}"
  }
}

here's the most important part, the attach_volume.sh, which will attach the volume, then partition it and format it if necessary.

#!/bin/sh
set -x
EC2_VOLUME_ID="${volume}"
EC2_INSTANCE_ID="`curl --silent http://169.254.169.254/latest/meta-data/instance-id`"
aws ec2 attach-volume --volume-id $EC2_VOLUME_ID --instance-id $EC2_INSTANCE_ID --device /dev/xvdh --region us-east-1
while [ ! -d /sys/block/xvdh ]; do sleep 1; done
if [ ! -d /sys/block/xvdh/xvdh1 ]; then
    echo -e "g\nn\np\n1\n\n\nw" | fdisk /dev/xvdh
    mkfs.ext4 /dev/xvdh1
fi

then we can use the standard cloud config to mount it

#cloud-config
---
mounts: 
 - [ xvdh, '/srv/nexus', 'auto', 'defaults,noexec', '0', '0' ]
runcmd: 
 - [ cloud-init-per, instance, chmod, chmod, 1777, /srv/nexus ] # make like temp, then docker can write whatever user directories in their
 - [ cloud-init-per, instance, docker, service, docker, start ] # start docker
 - [ cloud-init-per, instance, ecs, start, ecs ] # start ecs

How do I attach, format and mount a volume on boot in ecs?

1 Answers1