8

Edit: Tarun's answer does exactly what I asked for. Eugen's answer is also a very good solution. I ended up accepting Tarun's answer as correct, but using Eugen's. If you have a similar issue and are worried about other containers accessing the nginx status server, use Tarun's answer. If you'd rather stick to Docker's normal hostname scheme, use Eugen's.

+++ Original Question +++

I have an application that I build with docker-compose. I am trying to integrate monitoring through DataDog. I'm using DataDog's Agent container, and so far everything is working. I am trying to get nginx monitoring up and running by adapting this tutorial.

My application is defined in a docker-compose file like this:

version: '2'
services:
  flask:
    restart: always
    image: me/flask-app
    command: /home/app/flask/start_app.sh
    expose:
      - "8080"

  nginx:
    restart: always
    build: ./nginx
    command: /runtime/start_nginx.sh
    ports:
      - "80:80"
      - "443:443"
    expose:
      - "81"
    volumes:
      - app-static:/app-static:ro
    links:
      - flask:flask

  datadog-agent:
    image: me/datadog-agent
    env_file: ./datadog-agent/dev.env
    links:
        - flask
        - nginx
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /proc/mounts:/host/proc/mounts:ro
      - /sys/fs/cgroup:/host/sys/fs/cgroup:ro

Per the tutorial, I've added a server block to nginx that looks like this:

server {
  listen 81;

  location /nginx_status {
    stub_status on;
    access_log off;
    allow 127.0.0.1;
    deny all;
  }
}

With this configuration, I can check the nginx status from within the nginx container. So far, so good. Now I would like to change the "allow" directive in the location block to allow access to the datadog-agent service only. However, I don't know the IP of the datadog-agent. When configuring access to the Flask uwsgi server, I was able to use directives like this:

location / {
    uwsgi_pass: flask:8080;
}

But this doesn't seem to work for allow directives; if I try:

location /nginx_status {
    ...
    allow datadog-agent;
    ...
}

I get the following error:

nginx: [emerg] invalid parameter "datadog-agent" in /etc/nginx/sites-enabled/nginx-status:8

How can I safely expose the nginx status to my monitoring container?

nrlakin
  • 5,234
  • 3
  • 16
  • 27

3 Answers3

10

Think differently :)

Do bind a nginx-server (vhost) on port 10080 in addition - that server does offer the status location and what you need.

Server on 80/443 is also there and ONLY that one is bound/exposed to host ( exposed to the outer world ).

Since datadog is part of your docker-network / service network, it can still access 10080 in the internal network, but nobody else from the outer network.

Bulletproof, easy - no strings attached.

Eugen Mayer
  • 8,942
  • 4
  • 33
  • 57
  • 1
    Let me see if I understand. I remove the allow/deny directives from my server block. I listen on some high random port (I'm currently using 81, but it could be 10080). I expose the port (just as I am now, and not using "ports" directive). The (pseudo) port is accessible to all linked containers, but not to a physical port on the host. DataDog can access the stats, and so could my Flask container, but nothing outside can. I offload responsibility for protecting the data from nginx to docker, but I'm fine so long as no one gets into another linked container (then I'm screwed anyway). Correct? – nrlakin Jul 27 '17 at 19:26
  • exactly. Be sure you mean `expose` from the dockerfile, not as some say expose on the "host" ( thats not what you ment i think). We can say, 10080 is a private port to the containers. And you can choose any port you like, but highports make sense here, its a custom thing - you do not want to get into the way of others. If someone gets into your docker-server on the host, he is root and there is nothing you can protect against, right. But that is as safe as docker is, and most other linux things. – Eugen Mayer Jul 27 '17 at 19:31
  • 1
    I ended up accepting Tarun's answer, since it does exactly what I asked for, but I think I'll probably go this route as it seems less brittle to use Docker's DNS. I'm going to edit the question to credit your answer as well. – nrlakin Jul 27 '17 at 20:02
6

Since we are running the service through docker-compose and our issue being we don't know the IP of the agent. So the simple solution is to know the IP before starting. And that means assigning our agent a specific IP

Here is a update docker-compose to do that

version: '2'
services:
  flask:
    restart: always
    image: me/flask-app
    command: /home/app/flask/start_app.sh
    expose:
      - "8080"

  nginx:
    restart: always
    build: ./nginx
    command: /runtime/start_nginx.sh
    ports:
      - "80:80"
      - "443:443"
    expose:
      - "81"
    volumes:
      - app-static:/app-static:ro
    links:
      - flask:flask
    networks:
      agent:
        ipv4_address: 172.25.0.101
      default:

  datadog-agent:
    image: me/datadog-agent
    env_file: ./datadog-agent/dev.env
    links:
        - flask
        - nginx
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /proc/mounts:/host/proc/mounts:ro
      - /sys/fs/cgroup:/host/sys/fs/cgroup:ro
    networks:
      agent:
        ipv4_address: 172.25.0.100
networks:
  agent:
    driver: bridge
    ipam:
      config:
      - subnet: 172.25.0.0/24

Now you can do two possible things

server {
  listen 172.25.0.101:81;

  location /nginx_status {
    stub_status on;
    access_log off;
    allow 127.0.0.1;
    allow 172.25.0.100;
    deny all;
  }
}

You can listen only on 172.25.0.101 which is accessible only container running on agent network. Also you can add allow 172.25.0.100 to only allow the agent container to be able to access this.

Tarun Lalwani
  • 142,312
  • 9
  • 204
  • 265
4

There are two (easier) ways to go about it.

First one is docker-compose but since I already have a setup running since 2 years which doesn't use docker-compose, I went for the 2nd way.

Second way is Allow Directive with a range of IPs.

Eg:

    location /stub_status {
        stub_status;

        allow 172.18.0.0/16;   # This is my local docker IP range
        allow 192.168.0.0/16;  $ This is my production server IP range
        deny all;              # deny all other hosts   
 }

I am not security expert, but mostly 192.168.* IP range is for local networks, not sure about 172.18.* range though.

To get more idea about this IP range thing and CIDR stuff, refer below links http://nginx.org/en/docs/http/ngx_http_access_module.html

https://www.ripe.net/about-us/press-centre/understanding-ip-addressing

Niyojan
  • 544
  • 1
  • 6
  • 23
  • AFAICT Docker uses 172.16.0.0/12 (which includes 172.18.*), which is [reserved as a "Private Address Space"](https://tools.ietf.org/html/rfc1918), so this seems safe and great to me, thanks! – Don't Panic Mar 17 '21 at 21:28