Docker swarm route to containers on same node

Question

I have a 3 node swarm cluster, with each node having a haproxy ingress for TLS termination and a httpd server (deployed to swarm in global mode).

Haproxy uses httpd's service name to connect to a backend, but docker resolves (or routes via virtual IP) the service name to an arbitrary container instead of preferring the one on the same node.

Compose file

version: "3.9"
services:
  web_ingress:
    image: haproxy:alpine
    ports:
      - target: 80
        published: 80
        protocol: tcp
        mode: host
      - target: 443
        published: 443
        protocol: tcp
        mode: host
      - target: 8000
        published: 8000
        protocol: tcp
        mode: host
    volumes:
      - type: bind
        source: /home/haproxy-ingress
        target: /usr/local/etc/haproxy
        read_only: true
      - type: bind
        source: /etc/letsencrypt/archive
        target: /etc/letsencrypt/archive
        read_only: true
      - type: bind
        source: /etc/letsencrypt/live
        target: /etc/letsencrypt/live
        read_only: true
    # Running haproxy as non-root within the container makes SSL certificates inaccessible
    user: root
    deploy:
      mode: global

  frontpage:
    image: httpd:alpine
    expose:
      - 80
    volumes:
      - /home/httpd-frontpage/public-html:/usr/local/apache2/htdocs/
    deploy:
      mode: global

Haproxy config

global
    ssl-default-bind-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-bind-options prefer-client-ciphers no-sslv3 no-tlsv10 no-tlsv11 no-tlsv12 no-tls-tickets

    ssl-default-server-ciphersuites TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:TLS_CHACHA20_POLY1305_SHA256
    ssl-default-server-options no-sslv3 no-tlsv10 no-tlsv11 no-tlsv12 no-tls-tickets

    user root
    group root

defaults
    timeout connect 10s
    timeout client 30s
    timeout server 10s
    log global
    mode http
    option httplog
    maxconn 3000

frontend http-ingress
    bind *:80
    bind *:443 ssl crt /usr/local/etc/haproxy/fullchain.pem alpn h2,http/1.1
    redirect scheme https code 301 if !{ ssl_fc }

    # http-response set-header Strict-Transport-Security max-age=300

    default_backend frontpage

frontend stats
    bind *:8404
    stats enable
    stats uri /
    stats refresh 10s

backend frontpage
    server s0 frontpage:80 check

How do I configure this so that requests between haproxy and httpd will be served by httpd on the same node?

I know one way is to publish the httpd port in 'host' mode similar to haproxy, but I don't want to expose any httpd ports to the host's network.

score 0 · Answer 1 · answered Jan 18 '23 at 08:06

Dockers Service Mesh is not very complicated and does not offer this feature.

Your options are:

Use mode=host to publish the ports on the host on each node, and use "host.docker.internal:8500" (or whatever port) as the connection string to reach the port. Securing host mode ports so they only listen to localhost origin connections is possible but tricky and requires manual editing of the iptables config afaik.
You can also bypass Mesh Routing and directly use "tasks." to get all the dns entries for task replicas via a DNSRR response. The IPs will be overlay IPs, so determining which one is on the same docker node is an exercise for the reader.

Docker swarm route to containers on same node

1 Answers1