I've been trying to deploy a simple docker image to AWS ElasticBeanstalk using eb cli, but I can't get the docker service to start in ECS due to the following error:
invalid reference format: repository name must be lowercase
My Dockerrun.aws.json file:
{
"AWSEBDockerrunVersion": 2,
"containerDefinitions": [
{
"name": "ipsec-vpn-server",
"image": "hwdsl2/ipsec-vpn-server:latest",
"essential": true,
"memory": 128,
"portMappings": [
{
"hostPort": 500,
"containerPort": 500
},
{
"hostPort": 4500,
"containerPort": 4500
}
]
}
]
}
My .ebignore
# Only keep Dockerrun.aws.json
*
!Dockerrun.aws.json
I attached the AWSElasticBeanstalkMulticontainerDocker role to aws-elasticbeanstalk-service-role:
aws --profile my-profile-name \
iam attach-role-policy \
policy-arn arn:aws:iam::aws:policy/AWSElasticBeanstalkMulticontainerDocker \
--role-name aws-elasticbeanstalk-service-role
I created the aws eb environment like this:
export VPN_USER="some_secret_user"
export VPN_PASSWORD="some_secret_password"
export VPN_IPSEC_PSK="some_secret_ipsec_psk"
eb create vpn-example \
--profile my-profile-name \
--region us-east-1 \
--platform multi-container-docker-17.09.1-ce-\(generic\) \
--envvars VPN_USER=${VPN_USER} \
--envvars VPN_PASSWORD=${VPN_PASSWORD} \
--envvars VPN_IPSEC_PSK=${VPN_IPSEC_PSK} \
--cname some-vpn-example-domain
Environment did create correctly, but as you can see, it's not healthy ;)
The envionment creation runs for a while, creates load balencers, etc. But it fails after a while after trying to launch the task many times here in the ECS dashboard:
When clicking on a single task, I see this:
Status:
STOPPED (CannotPullContainerError: API error (400): invalid)
Details Status Reason:
CannotPullContainerError: API error (400): invalid reference format: repository name must be lowercase
Latest event shows following warning in elastic beanstalk environment:
Environment health has transitioned from Degraded to Severe. Command failed on all instances. ELB health is failing or not available for all instances.
eb logs has no output at all.
I don't use any uppercase letter in containerDefinitions's image. The only weird thing I found is the Image column in the task that says Dockerrun.aws.json where I would expect it to be hwdsl2/ipsec-vpn-server (I think).
I also thought this would have been a service and not a task. Any help would be greatly appreciated.
I can't seem to find a way to read the logs. I tried again from scratch and the logs for eb create command are the following:
WARNING: The Multi-container Docker platform requires additional ECS permissions. Add the permissions to the aws-elasticbeanstalk-ec2-role or use your own instance profile by typing "-ip {profile-name}".
For more information see: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/create_deploy_docker_ecs.html#create_deploy_docker_ecs_role
Creating application version archive "app-xxxx-xxxxxx_xxxxxx".
Uploading example-vpn/app-xxxx-xxxxxx_xxxxxx.zip to S3. This may take a while.
Upload Complete.
Environment details for: vpn-dev-01
Application name: example-vpn
Region: us-east-1
Deployed Version: app-xxxx-xxxxxx_xxxxxx
Environment ID: e-mtwbaums2n
Platform: arn:aws:elasticbeanstalk:us-east-1::platform/Multi-container Docker running on 64bit Amazon Linux/2.9.0
Tier: WebServer-Standard-1.0
CNAME: example-vpn-SNIP.us-east-1.elasticbeanstalk.com
Updated: 2018-03-22 18:55:03.173000+00:00
Printing Status:
INFO: createEnvironment is starting.
INFO: Using elasticbeanstalk-us-east-1-xxxxxxxxxxxx as Amazon S3 storage bucket for environment data.
INFO: Created security group named: sg-SNIP
INFO: Created security group named: awseb-e-SNIP
INFO: Created load balancer named: awseb-e-SNIP
INFO: Created Auto Scaling launch configuration named: awseb-e-SNIP
INFO: Created Auto Scaling group named: awseb-SNIP
INFO: Waiting for EC2 instances to launch. This may take a few minutes.
INFO: Created Auto Scaling group policy named: arn:aws:autoscaling:us-east-1:SNIP
INFO: Created Auto Scaling group policy named: arn:aws:autoscaling:us-east-1:SNIP
INFO: Created CloudWatch alarm named: awseb-e-SNIP
INFO: Created CloudWatch alarm named: awseb-e-SNIP
ERROR: Failed to start ECS task: arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is STOPPED.
ERROR: ECS task stopped due to: Essential container in task exited. (ipsec-vpn-server: )
ERROR: Failed to start ECS task: arn:aws:ecs:us-east-1:xxxxxxxxxxxx:task/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx is STOPPED.
ERROR: Failed to start ECS task after retrying 2 times.
ERROR: [Instance: i-xxxxxxxxxxxxxxxxx] Command failed on instance. Return code: 1 Output: trying 2 times.' --severity ERROR
+ exit 1.
Hook /opt/elasticbeanstalk/hooks/appdeploy/enact/03start-task.sh failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.
INFO: Command execution completed on all instances. Summary: [Successful: 0, Failed: 1].
ERROR: Create environment operation is complete, but with errors. For more information, see troubleshooting documentation.
I tried again with a different docker image nginx:latest and the service started so the problem I had was with hwdsl2/ipsec-vpn-server:latest directly. I tried again but locally:
docker pull hwdsl2/ipsec-vpn-server:latest
eb local run --debug
And it showed me the error:
ipsecvpnserver_1 | Error: This Docker image must be run in privileged mode.
ipsecvpnserver_1 |
ipsecvpnserver_1 | For detailed instructions, please visit:
ipsecvpnserver_1 | https://github.com/hwdsl2/docker-ipsec-vpn-server
ipsecvpnserver_1 |
elasticbeanstalk_ipsecvpnserver_1 exited with code 1
Logs in elasticbeanstalk aren't always easy to read. I added this to Dockerrun.aws.json:
"privileged": true,
Now it does start locally with success when running eb local run, but it's still failing in the cloud.
I've created an issue on hwdsl2/docker-ipsec-vpn-server
Related documentation: Multicontainer Docker Environments
