10

I'd like to use IMDSv2 inside a container running on an EC2 instance.

I want to use the tokens because they are required in my metadata options:

metadata_options {
  http_tokens   = "required"
  http_endpoint = "enabled"
}

Calling the API from the EC2 instance returns my token as expected.

curl -s -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"

However, if I try to call it from a docker container:

docker run -it curlimages/curl sh
/ $ curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"
curl: (56) Recv failure: Connection reset by peer

I just have a timeout.

According to this answer, it should work out of the box, but it's not. If I add a --network=host flag, it works, but that's not a solution for me.

Thanks

RobinFrcd
  • 4,439
  • 4
  • 25
  • 49

2 Answers2

24

I order to access IMDSv2 metadata from a docker container, you must increase the hop limit for IMDSv2 in the instance metadata configuration. From the aws docs:

In a container environment, if the hop limit is 1, the IMDSv2 response does not return because going to the container is considered an additional network hop. To avoid the process of falling back to IMDSv1 and the resultant delay, in a container environment we recommend that you set the hop limit to 2

To change the hop limit, you can use modify-instance-metadata-options in awscli:

aws ec2 modify-instance-metadata-options \
    --instance-id <instance_id> \
    --http-put-response-hop-limit 2 \
    --http-endpoint enabled
jordanm
  • 33,009
  • 7
  • 61
  • 76
  • 1
    Thank you so much for this great answer ! I was searching in the ECS doc, my bad ! Is putting a high value for `http-put-response-hop-limit` a bad practice, or it doesn't really matter ? – RobinFrcd Apr 15 '22 at 13:54
  • 1
    @RobinFrcd Not sure, I mostly run in EKS and have never needed to set higher than 2. – jordanm Apr 15 '22 at 14:00
  • 1
    @RobinFrcd The hop limit is set to 1 because the packet isn't supposed to leave localhost, so in this case setting it to 2 is necessary because it has to go from localhost to docker which is 2 hops. You don't want it higher than that though. More details in this blog post: https://aws.amazon.com/blogs/security/defense-in-depth-open-firewalls-reverse-proxies-ssrf-vulnerabilities-ec2-instance-metadata-service/ – Sami Samhuri Mar 08 '23 at 19:55
  • For anyone using ElasticBeanstalk see this GitHub issue for more details: https://github.com/aws/elastic-beanstalk-roadmap/issues/242 – Sami Samhuri Mar 08 '23 at 20:03
0

In case of in does not really work, you can try to increase the hop limit value.

Our context is: RKE2 + cilium on EC2 instances.

We have increase the hop limit from 2 to 3 and it works.

With hop limit=2

curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600"
^C
bash-4.2# curl http://169.254.169.254/latest/meta-data/
bash-4.2# curl http://169.254.169.254/latest/meta-data/ -vv
*   Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> GET /latest/meta-data/ HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.87.0
> Accept: */*
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 401 Unauthorized
< Content-Length: 0
< Date: Thu, 06 Apr 2023 09:02:51 GMT
< Server: EC2ws
< Connection: close
< Content-Type: text/plain
<
* Closing connection 0

After increased hop-limit=3

curl -X PUT "http://169.254.169.254/latest/api/token" -H "X-aws-ec2-metadata-token-ttl-seconds: 21600" -vv
*   Trying 169.254.169.254:80...
* Connected to 169.254.169.254 (169.254.169.254) port 80 (#0)
> PUT /latest/api/token HTTP/1.1
> Host: 169.254.169.254
> User-Agent: curl/7.87.0
> Accept: */*
> X-aws-ec2-metadata-token-ttl-seconds: 21600
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< X-Aws-Ec2-Metadata-Token-Ttl-Seconds: 21600
< Content-Length: 56
< Date: Thu, 06 Apr 2023 09:14:54 GMT
< Server: EC2ws
< Connection: close
< Content-Type: text/plain
<
* Closing connection 0
AQAEAPrjYxOT2_9q00Flibi5iB-KbE..redacted
Lamine BA
  • 129
  • 1
  • 8