Apache Mesos,MESOS-DNS, MARATHON and Docker

Question

In my environment running mesos-slave, mesos-master marathon and mesos-dns in standalone mode. I deployed mysql app to marathon to run as docker container.

MySql app configurations as follows.

{
    "id": "mysql",
    "cpus": 0.5,
    "mem": 512,
    "instances": 1,
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "mysql:5.6.27",
            "network": "BRIDGE",            
              "portMappings": [
                {
                  "containerPort": 3306,
                  "hostPort": 32000,
                  "protocol": "tcp"
                }
                ]
        }
    },
    "constraints": [
            [
              "hostname",
              "UNIQUE"
            ]],
    "env": {
        "MYSQL_ROOT_PASSWORD": "password"   
    },
      "minimumHealthCapacity" :0,
      "maximumOverCapacity" : 0.0
}

Then I deploy app called mysql client. Mysql client app needs to connect to mysql app.

mysql app config as follows.

{
    "id": "mysqlclient",
    "cpus": 0.3,
    "mem": 512.0,
    "cmd": "/scripts/create_mysql_dbs.sh",
    "instances": 1,
    "container": {
        "type": "DOCKER",
        "docker": {
            "image": "mysqlclient:latest",
            "network": "BRIDGE",            
            "portMappings": [{
                "containerPort": 3306,
                "hostPort": 0,
                "protocol": "tcp"
            }]
        }
    },
    "env": {
        "MYSQL_ENV_MYSQL_ROOT_PASSWORD": "password",
        "MYSQL_PORT_3306_TCP_ADDR": "mysql.marathon.slave.mesos.",
        "MYSQL_PORT_3306_TCP_PORT": "32000"     
    },
      "minimumHealthCapacity" :0,
      "maximumOverCapacity" : 0.0
}

My mesos-dns config.json. as follows

{
  "zk": "zk://127.0.0.1:2181/mesos",
  "masters": ["127.0.0.1:5050"],
  "refreshSeconds": 60,
  "ttl": 60,
  "domain": "mesos",
  "port": 53,
  "resolvers": ["127.0.0.1"],
  "timeout": 5,
  "httpon": true,
  "dnson": true,
  "httpport": 8123,
  "externalon": true,
  "listener": "127.0.0.1",
  "SOAMname": "ns1.mesos",
  "SOARname": "root.ns1.mesos",
  "SOARefresh": 60,
  "SOARetry":   600,
  "SOAExpire":  86400,
  "SOAMinttl": 60,
  "IPSources": ["mesos", "host"]
}
I can ping with service name mysql.marathon.slave.mesos. from host machine. But when I try to ping from mysql docker container I get host unreachable. Why docker container cannot resolve hsot name?

I tried with set dns parameter to apps. But its not work.

EDIT:

I can ping mysql.marathon.slave.mesos. from master/slave hosts. But I cannot ping from mysqlclient docker container. It says unreachable. How can I fix this?

Tobi · Answer 1 · 2016-02-29T15:26:40.997

0

Not sure what your actual question is, by guessing I think you want to know how you can resolve a Mesos DNS service name to an actual endpoint the MySQL client.

If so, you can use my mesosdns-resolver bash script to get the endpoint from Mesos DNS:

mesosdns-resolver.sh -sn mysql.marathon.mesos -s <IP_ADDRESS_OF_MESOS_DNS_SERVER>

You can use this in your create_mysql_dbs.sh script (whatever it does) to get the actual IP address and port where your mysql app is running.

You can pass in an environment variable like

"MYSQL_ENV_SERVICE_NAME": "mysql.marathon.mesos"

and then use it like this in the image/script

mesosdns-resolver.sh -sn $MYSQL_ENV_SERVICE_NAME -s <IP_ADDRESS_OF_MESOS_DNS_SERVER>

Also, please note that Marathon is not necessarily the right tool for running one-off operations (I assume you initialize your DBs with the second app). Chronos would be a better choice for this.

edited Feb 29 '16 at 15:26

answered Feb 29 '16 at 09:07

Tobi

31,405
8
58
90

My Question is , I can ping mysql.marathon.slave.mesos. form master/slave hosts. But I cannot ping from mysqlclient docker container. It says unreachable. – Manoj Gunawardena Feb 29 '16 at 14:39
well, please make yourself comfortable with host and bridged networking differences. The whole point of using a service discovery tool like Mesos DNS is that you don't need to specify fixed ip/ports for application, because the cluster decides where to put the application based on resources and/or contraints. See my edit. – Tobi Feb 29 '16 at 15:14
Thanks Tobi for the reply. This mesosdns-resolver.sh script is required to run dig in the container? – Manoj Gunawardena Feb 29 '16 at 15:22
It's a wrapper around dig basically... It parses the SRV record responses from Mesos DNS. You should also pay attention to use real ip addresses instead of just 127.0.0.1, because this will probably not work from within a container – Tobi Feb 29 '16 at 15:23
Tobi, Your script returns ip address and port of the mysql service. But my requirement is refer the mysql server with host name. At scaling If marathon starts a new mysql server and ip changed, I need to refer the mysql server with a host name. host name should not be changed. Can I do that using mesos-dns? – Manoj Gunawardena Feb 29 '16 at 16:38
Why by hostname? IP address should be perfectly fine. A hostname points to an actual IP address... – Tobi Feb 29 '16 at 17:07
When marathon scales and deploy new mysql DB instance ip address will change. Then need to change all applications accordingly – Manoj Gunawardena Feb 29 '16 at 17:30
That's why you use Mesos DNS. Please refer to the docs. I think you have a big misunderstanding here. – Tobi Feb 29 '16 at 17:51
mesos-dns will returns ip address of the current ip address running mysql. ip address can be change time to time. suppose now its 172.17.0.2. Next 172.17.0.3. Then I need to change apps configurations accordingly. Am I correct? – Manoj Gunawardena Feb 29 '16 at 19:28

Apache Mesos,MESOS-DNS, MARATHON and Docker

1 Answers1