Deploying docker container of Kibana 4 with port-mapping on Mesos/Marathon

Question

I'm using mesos and marathon to deploy a container of Kibana 4. The JSON to deploy is:

{
    "id": "/org/products/kibana/webapp",
    "instances": 1,
    "cpus": 1,
    "mem": 768,
    "uris": [],
    "constraints": [
        ["hostname", "UNIQUE"]
    ],
    "upgradeStrategy": {
        "minimumHealthCapacity": 0.5
    },
    "healthChecks": [
        {
            "protocol": "HTTP",
            "path": "/",
            "portIndex": 0,
            "initialDelaySeconds": 600,
            "gracePeriodSeconds": 10,
            "intervalSeconds": 30,
            "timeoutSeconds": 120,
            "maxConsecutiveFailures": 10
        }
    ],
    "env": {
        "ES_HOST":"172.23.10.23",
        "ES_PORT":"9200"
    },
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "myregistry.local.com:5000/org/kibana:4.0.0",
            "network": "BRIDGE",
            "portMappings": [
                {
                    "containerPort": 5601,
                    "hostPort": 0,
                    "servicePort": 50061,
                    "protocol": "tcp"
                }
            ]
        },
        "volumes": [
          {
            "containerPath": "/etc/localtime",
            "hostPath": "/etc/localtime",
            "mode": "RO"
          }
        ]
    }
}

But when I post it, the kibana app never wake up and the stderr log is:

I0227 12:22:44.666357  1149 exec.cpp:132] Version: 0.21.1
I0227 12:22:44.669059  1178 exec.cpp:206] Executor registered on slave 20150225-040104-1124079532-5050-952-S0

/kibana/src/index.js:58
      throw error;
            ^
Error: listen EADDRNOTAVAIL
  at errnoException (net.js:905:11)
  at Server._listen2 (net.js:1024:19)
  at listen (net.js:1065:10)
  at net.js:1147:9
  at asyncCallback (dns.js:68:16)
  at Object.onanswer [as oncomplete] (dns.js:121:9)

After that I try to eliminate a port mapping, because I found some references indicating that it's an port or network configuration problem. Then my Kibana 4 web app wake up fine, but I need configure a port-mapping to access via HTTP. I have not idea at about why marathon has problem with network and portMappings config. Some help will be appreciated.

score 1 · Answer 1 · answered Mar 02 '15 at 14:40

This is a nasty problem, and I encountered it as well (running Kibana 4 on Mesos + Marathon).

The short answer:

If you use current master of the Kibana repository, this won't happen - the relevant code has changed in the 4.1.0 snapshot which is master at the time of writing.

The long answer:

4.0.0 has this chunk of code in src/server/index.js:

var port = parseInt(process.env.PORT, 10) || config.port || 3000;
var host = process.env.HOST || config.host || '127.0.0.1';

Marathon supplies HOST and POST environment variables by default, and the HOST variable is set to the Mesos slave's hostname. The above code makes Kibana try to bind to the IP address of the Mesos slave (which Marathon has placed in HOST), which will fail, as it's running inside a Docker container.

If you want to run 4.0.0, you can just patch these lines to:

var port = config.port || 3000;
var host = config.host || '127.0.0.1';

The code looks like this in master at the moment.

Deploying docker container of Kibana 4 with port-mapping on Mesos/Marathon

1 Answers1