Haproxy setup with subdomain setup

Question

Hoping someone can help confirm if this suppose to work? I'm trying to route 3 subdomain traffic to the same haproxy host;

Here is my setup

haproxy with one interface ip 10.10.10.100 and dns name haproxy01.mydomain.com

3 CNAME records associated with it; sub1.mydomain.com, sub2.mydomain.com and sub3.mydomain.com

all the incoming traffic is for port 443.

There are two back end application servers that accepts traffic on three ports 8081, 8082, 8083, let say;

sub1.mydomain.com for 8081 sub2.mydomain.com for 8082 and sub3.mydomain.com for 8083

The application requires SSL pass through for only port 8081 traffic so I believe I've to use tcp mode for it the other traffic for 8082 and 8083 is also SSL but it can be terminated at the Haproxy but for the testing I went with all TCP mode.

My config section to achieve this is below;

    #Application Setup 
frontend mytraffic
    bind *:443
    mode  tcp
    acl host_sub1 hdr(host) -i sub1.mydomain.com
    acl host_sub2 hdr(host) -i sub2.mydomain.com
    acl host_sub3 hdr(host) -i sub3.mydomain.com

    use_backend sub1_nodes if host_sub1
    use_backend sub2_nodes if host_sub2
    use_backend sub3_nodes if host_sub3

    option tcplog backend sub1_nodes
    mode tcp
    balance roundrobin
    stick-table type ip size 200k expire 30m
    stick on src
    server node1 10.10.10.101:8081 check
    server node2 10.10.10.102:8081 check 
backend sub2_nodes
    mode tcp
    balance roundrobin
    stick-table type ip size 200k expire 30m
    stick on src
    server node1 10.10.10.101:8082 check
    server node2 10.10.10.102:8082 check 
backend sub3_nodes
    mode tcp
    balance roundrobin
    stick-table type ip size 200k expire 30m
    stick on src
    server node1 10.10.10.101:8083 check
    server node2 10.10.10.102:8083 check


    # APPLICATION SETUP END

When I try to access the appication servers via HAproxy for example for 8082 traffic it throws this is the logs;

localhost haproxy[6097]: x.x.x.x:51241 [20/Mar/2015:12:19:38.720] mytraffic mytraffic/ -1/-1/0 0 SC 0/0/0/0/0 0/0

appreciate some direction regarding this setup.

P.S. I can't embed any image for clarity here since it's my first post :-(

the log message was not complete; Mar 20 12:19:38 localhost haproxy[6097]: x.x.x.x:51241 [20/Mar/2015:12:19:38.720] mytraffic mytraffic/ -1/-1/0 0 SC 0/0/0/0/0 0/0 — Global Learning, Mar 20 '15 at 17:08
NOSRV means it couldn't find a suitable backend. Can you access the stats page or socket and see what backends HAproxy thinks are up? — Jim G., Mar 20 '15 at 17:20
Also check if there is connectivity between haproxy and the application servers. If using nc: nc -v 10.10.10.101 8081, nc -v 10.10.10.102 8081 , etc. — hdanniel, Mar 20 '15 at 18:10
The lastcheck on the haproxy monitoring dashboard is reporting all nodes up. I've also confirmed connectivity from the Haproxy console to these application servers on all ports. — Global Learning, Mar 20 '15 at 18:12

score 12 · Accepted Answer · answered Mar 20 '15 at 18:12

With TCP mode, HAProxy won't decode the HTTP request, so your acl lines won't do anything and the frontend will never be able to match a backend, as shown by the logs you entered: mytraffic/<NOSRV> means it wasn't able to pick a backend or server.

You'd have to split the 3 subdomains into 2 different frontends, each with their own IPs since they're all connecting on port 443. One for passthrough, the other for the SSL termination and content switching using mode http. The caveat here being that if you were to add a 4th subdomain (sub4.mydomain.com) that also required passthrough, you'd then need a 3rd frontend and IP.

You'd also need to create different CNAME or A records in DNS so that the right subdomains point to the right IPs.

Given this DNS config:

10.10.10.100        A         haproxy01-cs.mydomain.com
10.10.10.101        A         haproxy01-pt1.mydomain.com
10.10.10.102        A         haproxy01-pt2.mydomain.com
sub1.mydomain.com   CNAME     haproxy01-pt1.mydomain.com
sub2.mydomain.com   CNAME     haproxy01-cs.mydomain.com
sub3.mydomain.com   CNAME     haproxy01-cs.mydomain.com
sub4.mydomain.com   CNAME     haproxy01-pt2.mydomain.com

The HAproxy config would look something like this:

#Application Setup 
frontend ContentSwitching

  bind 10.10.10.100:443
  mode  http
  option httplog
  acl host_sub2 hdr(host) -i sub2.mydomain.com
  acl host_sub3 hdr(host) -i sub3.mydomain.com
  use_backend sub2_nodes if host_sub2
  use_backend sub3_nodes if host_sub3

frontend PassThrough1
  bind 10.10.10.101:443
  mode  tcp
  option tcplog
  use_backend sub1_nodes     

frontend PassThrough2
  bind 10.10.10.102:443
  mode  tcp
  option tcplog
  use_backend sub4_nodes

backend sub1_nodes
  mode tcp
  balance roundrobin
  stick-table type ip size 200k expire 30m
  stick on src
  server node1 10.10.10.101:8081 check
  server node2 10.10.10.102:8081 check 

backend sub2_nodes
  mode http
  balance roundrobin
  stick-table type ip size 200k expire 30m
  stick on src
  server node1 10.10.10.101:8082 check
  server node2 10.10.10.102:8082 check 

backend sub3_nodes
  mode http
  balance roundrobin
  stick-table type ip size 200k expire 30m
  stick on src
  server node1 10.10.10.101:8083 check
  server node2 10.10.10.102:8083 check

backend sub4_nodes
  mode tcp
  balance roundrobin
  stick-table type ip size 200k expire 30m
  stick on src
  server node1 10.10.10.101:8084 check
  server node2 10.10.10.102:8084 check

Actually, the IPs I have here don't make any sense (I used 10.10.10.101-102 for the front-ends where you've got them assigned to the nodes), but you get the idea of how it would look. — GregL, Mar 20 '15 at 18:15
@GreL - THANK You. I'm going to try this and circled with results. I was worried about the tcp mode and acl use before using it. — Global Learning, Mar 20 '15 at 18:16
Thanks GregL. it works. One last question if I may? to have SSL from client --> haproxy and to haproxy --> backend server (like it can terminate but start a new one to the backend), what's the recommended way of doing it? ssl pass through with tcp mode? The reason I wanted the passthrough for port 8081 traffic is because of the requirement to have mutually authenticated session with application and client. The rest of the ports 8082 and 8083 traffic requires ssl but it doesn't require mutual authentication. (right now I went with passthrough approach). — Global Learning, Mar 20 '15 at 19:21
Well, HAProxy can do the client certificate checking by using the `ca-file` and `verify optional` options to the `bind` statement. If you did that, you could have a single frontend bound to one IP, and then content switch to the appropriate backends as required by the _host_ header. To confirm that mutual authentication worked you can create an acl like `acl ClientSSLValid ssl_c_verify 0`, then add it as another condition to the `use_backend` statement like `use_backend sub1_nodes if host_sub1 ClientSSLValid` — GregL, Mar 20 '15 at 20:03
There's decent write-ups [here](http://blog.haproxy.com/2012/10/03/ssl-client-certificate-management-at-application-level/) and [here](https://raymii.org/s/tutorials/haproxy_client_side_ssl_certificates.html) on how to do this, with more detail than I've provided. — GregL, Mar 20 '15 at 20:07
Thanks GregL. I'll read the links to play with...appreciate all the help on this problem. — Global Learning, Mar 20 '15 at 20:44

Haproxy setup with subdomain setup

1 Answers1