Unable to mount EBS drives to an Ubuntu box using AWS CloudFormation (UserData not working)

Question

I have an otherwise working CloudFormation template that creates an AWS instance 2 EBS volumes. The volumes are visible on the machine via lsblk. I am just having trouble getting them formatted and mounted. It's as if the UserData script is not running at all.

It's an Ubuntu 14.04 machine. Any idea what I might be doing wrong?

EDIT: added full template for reference

{
    "AWSTemplateFormatVersion": "2010-09-09",
    "Description": ".",
    "Parameters": {
        "KeyName": {
            "Description": "Name of an existing EC2 KeyPair to enable SSH access to the instance",
            "Type": "AWS::EC2::KeyPair::KeyName",
            "ConstraintDescription": "must be the name of an existing EC2 KeyPair.",
        },
        "InstanceType": {
            "Description": "EC2 instance type",
            "Type": "String",
            "Default": "t2.medium",
            "AllowedValues": [
                "t2.micro",
                "t2.small",
                "t2.medium",
                "c3.large",
                "c3.xlarge",
                "c4.large",
                "c4.xlarge"
            ],
            "ConstraintDescription": "must be a valid EC2 instance type."
        },
        "VpcId": {
            "Type": "AWS::EC2::VPC::Id",
            "Description": "VpcId of your existing Virtual Private Cloud (VPC)",
        },
        "SubnetId": {
            "Type": "AWS::EC2::Subnet::Id",
            "Description": "Existing Subnet ID",
        }
    },
    "Mappings": {
        "AWSInstanceType2Arch": {
            "t2.micro": {
                "Arch": "HVM64"
            },
            "t2.small": {
                "Arch": "HVM64"
            },
            "t2.medium": {
                "Arch": "HVM64"
            },
            "c3.large": {
                "Arch": "HVM64"
            },
            "c3.xlarge": {
                "Arch": "HVM64"
            },
            "c4.large": {
                "Arch": "HVM64"
            },
            "c4.xlarge": {
                "Arch": "HVM64"
            }
        },
        "AWSRegionArch2AMI": {
            "us-west-2": {
                "HVM64": "ami-7ba1b34b"
            }
        }
    },
    "Resources": {
        "EC2Instance": {
            "Type": "AWS::EC2::Instance",
            "Properties": {
                "InstanceType": {
                    "Ref": "InstanceType"
                },
                "KeyName": {
                    "Ref": "KeyName"
                },
                "SecurityGroupIds": [
                    {
                        "Ref": "MySecurityGroup"
                    }
                ],
                "SubnetId": {
                    "Ref": "SubnetId"
                },
                "ImageId": {
                    "Fn::FindInMap": [
                        "AWSRegionArch2AMI",
                        {
                            "Ref": "AWS::Region"
                        },
                        {
                            "Fn::FindInMap": [
                                "AWSInstanceType2Arch",
                                {
                                    "Ref": "InstanceType"
                                },
                                "Arch"
                            ]
                        }
                    ]
                },
                "Tags": [
                    {
                        "Key": "Name",
                        "Value": "grafana"
                    }
                ],
                "UserData": {
                    "Fn::Base64": {
                        "Fn::Join": ["",
                            [
                            "sudo mkfs -t ext4 /dev/xvdf\n",
                            "sudo mkfs -t ext4 /dev/xvdg\n",
                            "sudo mkdir /mnt/influx /mnt/db\n",
                            "sudo mount /dev/xvdf /mnt/influx\n",
                            "sudo mount /dev/xvdg /mnt/db\n",
                            "sudo echo \"/dev/xvdf /mnt/influx ext4 defaults,nofail 0 2\" >> /etc/fstab\n",
                            "sudo echo \"/dev/xvdg /mnt/db ext4 defaults,nofail 0 2\" >> /etc/fstab\n",
                            "sudo wget http://influxdb.s3.amazonaws.com/influxdb_0.9.4.1_amd64.deb\n",
                            "sudo dpkg -i influxdb_0.9.4.1_amd64.deb\n",
                            "sudo /etc/init.d/influxdb start\n",
                            "sudo wget https://grafanarel.s3.amazonaws.com/builds/grafana_2.1.3_amd64.deb\n",
                            ]
                        ]
                    }
                }
            }
        },
        "MySecurityGroup": {
            "Type": "AWS::EC2::SecurityGroup",
            "Properties": {
                "VpcId": {
                    "Ref": "VpcId"
                },
                "GroupDescription": "Security group instace",
                "SecurityGroupIngress": [
                    {
                        "IpProtocol": "tcp",
                        "FromPort": "22",
                        "ToPort": "22",
                        "CidrIp": "10.0.0.0/16"
                    }
                ]
            }
        },
        "EBSVolume1": {
            "Type": "AWS::EC2::Volume",
            "Properties": {
                "Size": 50,
                "AvailabilityZone": {
                    "Fn::GetAtt": [
                        "EC2Instance",
                        "AvailabilityZone"
                    ]
                }
            }
        },
        "EBSVolume2": {
            "Type": "AWS::EC2::Volume",
            "Properties": {
                "Size": 200,
                "AvailabilityZone": {
                    "Fn::GetAtt": [
                        "EC2Instance",
                        "AvailabilityZone"
                    ]
                }
            }
        },
        "EBSVolumeMount1": {
            "Type": "AWS::EC2::VolumeAttachment",
            "Properties": {
                "InstanceId": {
                    "Ref": "EC2Instance"
                },
                "VolumeId": {
                    "Ref": "EBSVolume1"
                },
                "Device": "/dev/sdf"
            }
        },
        "EBSVolumeMount2": {
            "Type": "AWS::EC2::VolumeAttachment",
            "Properties": {
                "InstanceId": {
                    "Ref": "EC2Instance"
                },
                "VolumeId": {
                    "Ref": "EBSVolume2"
                },
                "Device": "/dev/sdg"
            }
        }
    },
    "Outputs": {
        "InstanceId": {
            "Description": "InstanceId of the newly created EC2 instance",
            "Value": {
                "Ref": "EC2Instance"
            }
        }
    }
}

UserData is only run on the **initial** boot, not subsequent reboots. Are you terminating the instance each time, or just rebooting? Also, your template doesn't mention any EBS volumes, only an instance. Can you post or gist the full template? — Craig Watson, Sep 29 '15 at 21:30
@CraigWatson: updated with the full template. No, I only run (and want to run) UserData at first boot. Basically, mount volumes, then install a package — Michael Manoochehri, Sep 29 '15 at 23:36
Can you mount them manually in the instance? Have you tried checking the guide here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ebs-using-volumes.html — Craig Watson, Sep 30 '15 at 09:08

score 6 · Answer 1 · answered Sep 30 '15 at 09:51

Here's the answer - the EBS needs to be attached fully.

"UserData": {
                    "Fn::Base64": {
                        "Fn::Join": ["",
                            [
                                "#!/bin/bash\n",
                                "## Wait for EBS mounts to become available\n",
                                "while [ ! -e /dev/xvdf ]; do echo waiting for /dev/xvdf to attach; sleep 10; done\n",
                                "while [ ! -e /dev/xvdg ]; do echo waiting for /dev/xvdg to attach; sleep 10; done\n",
                                "sudo mkfs -t ext4 /dev/xvdf\n",
                                "sudo mkfs -t ext4 /dev/xvdg\n",
                                "sudo mkdir /mnt/influx /mnt/db\n",

I suppose an alternative solution, depending on what you need to do, is create an AMI image that has the EBS mounts formatted already. — Michael Manoochehri, Oct 09 '15 at 18:13

Unable to mount EBS drives to an Ubuntu box using AWS CloudFormation (UserData not working)

1 Answers1