How can I get the IP address of a unit started on a machine and use it in another unit with fleet?

Question

I'm new to CoreOS and Docker and I'm facing with a problem with fleet.

I have a standard unit launching a POSTGRES container and I would like to know the IP address of the machine where this unit is started. I have actually a cluster of 3 machines and the POSTGRES unit isn't always started on the same machine (which means the IP is not static).

I need it when I start another unit (see below), which needs a POSTGRES.

I'm using at the moment the Unit Parameter called BindsTo :

[Unit]
Description=Test
After=docker.service
After=postgres@1.service

Requires=docker.service
Requires=postgres@1.service

BindsTo=postgres@1.service

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill test%i
ExecStartPre=-/usr/bin/docker rm test%i
ExecStart=/usr/bin/docker run -rm --name test%i -e HOST="HereThePostgresIP" sryther/test
ExecStop=/usr/bin/docker stop test%i

N.B. : This unit doesn't need to be started on the same machine as the POSTGRES unit.

I tried with %H variable but it returns the hostname and not the IP address of the machine.

Flannel is also used in my cluster.

Thanks!

I found a workaround :

[Unit]
Description=Test
After=docker.service

Requires=docker.service

[Service]
TimeoutStartSec=0
ExecStartPre=-/usr/bin/docker kill test%i
ExecStartPre=-/usr/bin/docker rm test%i

ExecStart=/usr/bin/docker run -rm --name test%i --link postgres1:postgres sryther/test
ExecStop=/usr/bin/docker stop test%i

And the IP address is set in the container environment as POSTGRES_PORT_5432_TCP_ADDR.

score 4 · Answer 1 · answered Mar 13 '15 at 16:08

4

I am using registrator for discovery in my fleet environment. When I start up my fleet I modify the cloud-config file to also include registrator (along with etcd, flannel, fleet, etc). Registrator camps on docker and recognizes when new containers are started/killed/stop. Registrator manages a database of discovered containers, it does so in several flavors (like consul, skydns). I configure mine to run in skydns mode. So, here is an example of me starting up my postgres server with a unit file, I don't know where it is going to land:

[Unit]
Description=Postgres
After=docker.service
Requires=flanneld.service docker.service etcd.service

[Service]
Restart=always
ExecStartPre=-/usr/bin/env docker kill postgresql
ExecStartPre=-/usr/bin/env docker rm postgresql
ExecStartPre=/usr/bin/env docker pull sameersbn/postgresql:9.4
ExecStart=/usr/bin/docker run --name postgresql sameersbn/postgresql:9.4
ExecStop=-/usr/bin/docker stop postgresql

When postgres starts, I get the etcd updated with a postgres entry:

/skydns/net/tacodata/postgresql/aup1:postgresql:5432

And if I fetch that one, I';; see the ip and port:

$ etcdctl get /skydns/net/tacodata/postgresql/aup1:postgresql:5432
{"host":"10.1.43.5","port":5432}

You could modify your application to do that. Or, you can run the container skydns which automatically updates dns for you, so you have an SRV record and an A record. In my installation I use a test domain called tacodata.net, so, after postgresql comes up, I have dns records!

root@f7e403be967a:/# host -t srv postgresql.tacodata.net 10.1.45.1
Using domain server:
Name: 10.1.45.1
Address: 10.1.45.1#53
Aliases: 

postgresql.tacodata.net has SRV record 10 100 5432
aup1:postgresql:5432.postgresql.tacodata.net.

When I start my dependent applications, I make them require postgresql.service, and I simply reference ip address to the application using postgresql.tacodata.net.

-g

answered Mar 13 '15 at 16:08

Greg

6,571
2
27
39

I see how your solution works but I can't find a way to implement it in my cluster. I'm trying to add skydns and registrator services in my cloud-config but it seems I'm not writing it well. Here is my current configuration [on pastebin](http://pastebin.com/Q2nHp7ia), do you see anything misconfigured ? – Paul Rey Mar 20 '15 at 08:33
I note that you are referencing one of my images (tacodata/*), you are welcome to do that, know that they disappear every now and then! – Greg Mar 20 '15 at 11:42
I suggest some experimenting to figure out what exactly registrator/skydns can do for you. It should be pretty easy to simply use docker to add registrator. Make sure it works (once you add registrator you can use etcdctl to see if it is updating skydns namespace when containers come and go). Once you have that working, you can experiment with skydns (picki a domain name you want skydns to use, in my case, skydns used tacodata.net.) Once you can get dig or nslookup working on skydns the final experiment would be with modifying docker boot to use skydns name service automatically (--dns). – Greg Mar 20 '15 at 11:52
Once you have these things working then you can go about the task of figuring out how to orchestrate the unit file startup / dependency stuff. – Greg Mar 20 '15 at 11:57
Ok, I changed tacodata/skydns-coreos to skynetservices/skydns. – Paul Rey Mar 20 '15 at 13:05
Here's my [new cloud-config](http://pastebin.com/Hxptdnpy) and I have this output when I do a journalctl -u skydns.service : "skydns: falling back to default configuration, could not read from etcd: Get http://172.17.22.0:4001/v2/keys/skydns/config?quorum=false&recursive=false&sorted=false: dial tcp 172.17.22.0:4001: network is unreachable". – Paul Rey Mar 20 '15 at 13:06
And my ifconfig : flannel0: inet 172.17.22.0. When I curl this address I have a response from my etcd (so I think it works well at this point). – Paul Rey Mar 20 '15 at 13:08
I'm trying to experiment what you suggested right now. I'll post my results here, even if I would prefer to move this discussion to chat but I do not have enough reputation on stackoverflow.. – Paul Rey Mar 20 '15 at 13:12
good luck. once you figure out all of the moving parts it will become clear :-) I didn't know that SO could do a chat. – Greg Mar 20 '15 at 13:51

Paul Rey · Accepted Answer · 2015-03-26T16:29:16.800

According to Greg I used SkyDNS and Registrator :

I am using registrator for discovery in my fleet environment. When I start up my fleet I modify the cloud-config file to also include registrator (along with etcd, flannel, fleet, etc). Registrator camps on docker and recognizes when new containers are started/killed/stop. Registrator manages a database of discovered containers, it does so in several flavors (like consul, skydns). I configure mine to run in skydns mode.

Below you can find my working cloud-config, keep in mind that I'm using CoreOS 607 and my tests are performed on VMware. My goal was to set up 3 machines to make a small cluster with one etcd running on each machine like this.

I managed to found this solution thanks to Greg on this topic and with this cloud-config file used with Vagrant.

I hope it'll help :)

#cloud-config

coreos:
  etcd:
    discovery: https://discovery.etcd.io/4c8c3f9a67048e76e9075c97d9d63c0c
    addr: 192.168.1.13:4001
    peer-addr: 192.168.1.13:7001
  fleet:
    public-ip: 192.168.1.13
  units:
    - name: etcd.service
      command: start
    - name: flanneld.service
      drop-ins:
        - name: 50-network-config.conf
          content: |
            [Service]
            ExecStartPre=/usr/bin/etcdctl set /coreos.com/network/config '{ "Network": "172.16.0.0/16" }'
      command: start
    - name: fleet.service
      command: start
    - name: skydns.service
      enable: false
      content: |
        [Unit]
        Description=Skydns
        After=docker.service
        After=fleet.service
        [Service]
        ExecStartPre=/usr/bin/docker pull skynetservices/skydns
        ExecStart=/usr/bin/env bash -c '/usr/bin/docker run -p 53:53 --name skydns -e ETCD_MACHINES="http://$(ifconfig docker0 | awk \'/\<inet\>/ { print $2}\'):4001" skynetservices/skydns -nameservers="8.8.8.8:53" --addr="0.0.0.0:53"'
        ExecStop=/usr/bin/docker stop skydns
        Restart=always

        [X-Fleet]
        Global=true

        [Install]
        WantedBy=multi-user.target
    - name: registrator.service
      enable: false
      content: |
        [Unit]
        Description=Registrator
        After=docker.service
        After=fleet.service

        [Service]
        ExecStartPre=/usr/bin/docker pull gliderlabs/registrator
        ExecStart=/usr/bin/env bash -c '/usr/bin/docker run --net=host -p 8080:8080 -p 8443:8443 -v /var/run/docker.sock:/tmp/docker.sock --name registrator gliderlabs/registrator skydns2://$(ifconfig docker0 | awk \'/\<inet\>/ { print $2}\'):4001/skydns.local'
        ExecStop=/usr/bin/docker stop registrator
        Restart=always

        [X-Fleet]
        Global=true
    - name: primordial.service
      command: start
      content: |
        [Unit]
        Description=Load and start fleet services
        After=fleet.service
        Requires=fleet.service
        After=etcd.service
        Requires=etcd.service
        After=flanneld.service
        Requires=flanneld.service

        [Service]
        Type=oneshot
        RemainAfterExit=yes
        ExecStartPre=/usr/bin/systemctl stop docker.service
        ExecStartPre=/usr/bin/ifconfig docker0 down
        ExecStartPre=/usr/bin/brctl delbr docker0
        ExecStartPre=/usr/bin/systemctl start docker.service
        ExecStart=/usr/bin/fleetctl start /etc/systemd/system/skydns.service
        ExecStart=/usr/bin/fleetctl start /etc/systemd/system/registrator.service
ssh_authorized_keys:
  - ssh-rsa AAAA....

How can I get the IP address of a unit started on a machine and use it in another unit with fleet?

2 Answers2

Linked