Multi-container docker on AWS - Nginx use host machine /etc/hosts resolver

Question

I have a multi-container docker environment on Amazon Elastic Beanstalk with the following Dockerrun.aws.json file:

{ 
    "AWSEBDockerrunVersion": 2, 
    "containerDefinitions": [ 
      { 
        "name": "web", 
        "memoryReservation": 256, 
        "image": "my/nginx/repo/image",  
        "portMappings": [ 
          { 
            "hostPort": 80, 
            "containerPort": 80 
          } 
        ], 
        "links": [ 
          "api" 
        ], 
        "essential": true 
      }, 
      { 
        "name": "api", 
        "memoryReservation": 256, 
        "image": "my-api/repo", 
        "essential": true, 
        "portMappings": [ 
          { 
            "hostPort": 3000, 
            "containerPort": 80 
          } 
        ]
      } 
    ] 
  }

Ultimately I want the node app served by nginx to resolve requests to named addresses from linked containers, so in my web image (node app) I'd like to make a request to http://api/some/resource and let nginx resolve that to the api container.

Now, since docker adds a host entry for the api container due to the specified link, I want the nginx server to resolve addresses from the hosts etc/hosts file, however as I found out, nginx uses it's own resolver. After researching the issue a bit I found out that in non-Elastic Beanstalk multi-container solutions and with user-defined networks, the resolver would be provided by docker on 127.0.0.11, however since it is currently not possible to define user-defined networks in the Dockerrun.aws.json, I keep looking for a different solution. The links can be resolved inside the container, pinging api does work, however, nginx does it's own thing there.

I have read about dnsmasq as well, however, I wanted to get this running without installing this package, do I even have a choice here ?

I saw in doc The default Docker network mode is bridge. but some how nginx if not able to resolve my backend server using 127.0.0.11 as dns in AWS env, local its working fine. — sandeep, Apr 16 '18 at 15:01
@sandeep Elastic beanstalk should create entries for your links specified in your dockerrun.aws.json file. I have done it before. Make sure you add links between containers and you can reference them by the container name. If you need help, please create a question with your config and link it here in the comments. — the_critic, Apr 18 '18 at 21:55
yes you are correct links are creating entry in hosts file and i am able to ping memcache server from nginx container. But nginx is not able to resolve. — sandeep, Apr 20 '18 at 09:12
@sandeep Is your nginx in a container as well ? I had my nginx as a container as well and proxied to other containers. — the_critic, Apr 20 '18 at 09:39

Luca Gibelli · Answer 1 · 2018-04-18T20:05:32.307

1

There is no way to force nginx to use the entries from /etc/hosts.

You can however use a map { } in your nginx config to tell nginx how to convert hostnames to IPs. You would need a script to convert your /etc/hosts to a format that can be used in a map, i.e. hostname ip vs ip hostname.

Here is an example map:

map $container_hostname $container_ip {
    default 127.0.0.1;
    containerA X.X.X.X;
    containerB Y.Y.Y.Y;
}

Later in the config file you can do:

server_name   ~^(www\.)?(?<container_hostname>.+)$;

location / {
    proxy_pass http://$container_ip:80;
}

nginx will match the requested server_name and store it in $container_hostname:

http://nginx.org/en/docs/http/server_names.html#regex_names

Then it will look up the hostname in the map, obtain the corresponding IP and pass it to proxy_pass.

More info on map: http://nginx.org/en/docs/http/ngx_http_map_module.html

edited Apr 18 '18 at 20:05

answered Apr 18 '18 at 17:25

Luca Gibelli

2,731
1
22
30

Problem with this solution is that I don't know the container IP. – sandeep Apr 20 '18 at 09:04
I'm confused, you wrote that you want the nginx server to resolve addresses from /etc/hosts, so I assumed you already have the ip->host mapping there. What am I missing? – Luca Gibelli Apr 20 '18 at 13:13
1

The mapping in /etc/hosts is autogenerated by docker. – ei-grad Apr 20 '18 at 14:24
@sandeep in case it's not clear: $container_ip is automatically filled by nginx using the content of the map stanza. The process goes like this: 1) nginx reads the hostname in the "Host: " header sent by the http client and stores it in the $container_hostname var 2) nginx looks up $container_hostname in the map { } stanza and fills $container_ip with the corresponding IP 3) nginx does a proxy_pass to that IP – Luca Gibelli Apr 20 '18 at 17:16

ei-grad · Answer 2 · 2018-04-19T20:40:28.567

0

No nginx - no problem. Just replace it with an application load balancer. And what about requests inside the linked containers? They will go directly to the linked containers anyway, bypassing the nginx, so it is useless here. You can use port 80 inside the containers, and put them on different host ports to connect the load balancer.

edited Apr 19 '18 at 20:40

answered Apr 19 '18 at 20:34

ei-grad

174
5

I have some content based requirement to route the traffic. thats why i need nginx-lua, openresty. – sandeep Apr 20 '18 at 09:09

Multi-container docker on AWS - Nginx use host machine /etc/hosts resolver

2 Answers2