10

Our application requires cookie based sticky sessions, so we want to use HAproxy to balance incoming traffic towards a farm of IIS servers.

We are using the following config which seems to work on the lab (round-robin working fine and session preserved), but fails when applied in producion with more that 3k concurrent users:

frontend Front_http

bind :80
mode http
default_backend backend_http
stats enable
capture cookie ASP.NET_SessionId len 32
maxconn 10000

frontend Front_https

mode http
default_backend backend_https
bind *:443 ssl crt /etc/haproxy/cert.pem 
capture cookie ASP.NET_SessionId len 32
maxconn 10000

backend backend_http

 balance roundrobin
 option forwardfor
 stick-table type ip size 20k expire 5m
 appsession ASP.NET_SessionId len 64 timeout 5m request-learn prefix
 server Server_1 192.168.10.81:80 cookie Server_1
 server Server_2 192.168.10.81:80 cookie Server_2
 server Server_3 192.168.10.81:80 cookie Server_3

backend backend_https

 balance roundrobin
 option forwardfor
 stick-table type ip size 20k expire 5m
 appsession ASP.NET_SessionId len 64 timeout 5m request-learn prefix
 server Server_1 192.168.10.81:80 cookie Server_1 ssl verify none
 server Server_2 192.168.10.81:80 cookie Server_2 ssl verify none
 server Server_3 192.168.10.81:80 cookie Server_3 ssl verify none
 http-request set-header X-Forwarded-Port %[dst_port]
 http-request add-header X-Forwarded-Proto https if { ssl_fc }

From the HAProxy 1.5.8 documentation I understand cookie based stickiness is achieved with command "appsession", but I don't understand the role other commands play, like "capture cookie" or "stick-table", are they necessary at all when using appsession? Can anyone help me understand how they work, and advise if you detect anything wrong with our config.

Vanessa Bullock
  • 101
  • 1
  • 1
  • 3

2 Answers2

29

First of all, could you explain what "does not work" or which problems are you facing with your current configuration?

There are a few issues in your current configuration: - appsession stickness does not resist to a reload. It means stickiness is lost after each reload of HAProxy - you may have a typo in your SSL backend, since you're forwarding SSL traffic to port 80, which is the same port you used for clear HTTP.

HAProxy allows many ways to do cookie based persistence.

  • cookie insertion: HAProxy set up itself a cookie:

    backend mybk
  [...]
  cookie SERVERID insert indirect nocache
  [...]
  server s1 10.0.0.1:80 check cookie s1
  server s2 10.0.0.2:80 check cookie s2

  • cookie prefix: HAProxy uses an existing cookie 'usually application one) and prefix its value by the server name:

    backend mybk
  [...]
  cookie ASP.NET_SessionId prefix nocache
  [...]
  server s1 10.0.0.1:80 check cookie s1
  server s2 10.0.0.2:80 check cookie s2

  • stick table: HAProxy learn and use the application cookie, without modifying it:

    backend mybk
  [...]
  stick-table type string len 64 size 100k expire 15m
  stick store-response res.cookie(ASP.NET_SessionId)
  stick match req.cookie(ASP.NET_SessionId)
  [...]
  server s1 10.0.0.1:80 check 
  server s2 10.0.0.2:80 check

Note: you should use a peers section to keep the data synchronized between 2 HAProxys and when reloading the configuration Note2: the expire parameter should match your application cookie timeout

Last but not least, HAProxy will report you flags about cookie based persistence (understand the one with the cookie keyword) in your log lines. That way, you'll know the status of the request (was there a cookie, was it valid, etc...) and the action taken by HAProxy (insert a new cookie, etc...)

You can have a look at this blog page to get more information about HAProxy: http://blog.haproxy.com/2012/03/29/load-balancing-affinity-persistence-sticky-sessions-what-you-need-to-know/

Baptiste

Baptiste
  • 1,686
  • 13
  • 6
  • thanks so much for your complete explanation. By "not working" I meant browsing slowness and eventually clients getting http errors 504 and 500. We monitored balancing of newly initiated sessions and it looked even, however at a certain point one of the IISs seemed to have a lot more sessions than the others and stopped responding, and was followed by another IIS later on. I have to say, though, that our platform is in Azure and we later learnt that there was being a severe outage in Azure probably affecting the IIS's, so perhaps this wasn't about the HAproxy after all. – Vanessa Bullock Nov 26 '14 at 10:15
  • I am however reviewing my configuration as some things look odd to me in the light of your explanations, I will stick to one of the methods you suggest to achieve stickyness and will test again only when Azure issues are really resolved. Also because this is implemented in Azure, there are network limitations that make the high availability topology impossible to implement. – Vanessa Bullock Nov 26 '14 at 10:15
  • 504 means the server was too slow to answer. 'timeout server' is the key here. 500 are generated by your application server, HAProxy can't fix them :) – Baptiste Nov 27 '14 at 00:10
  • I tried your third suggestion: stick-table type string len 64 size 100k expire 15m stick store-response res.cookie(ASP.NET_SessionId) stick match req.cookie(ASP.NET_SessionId) but the HAProxy doesn't like it, I get an error when reloading. I am finally using "cookie ASP.NET_SessionId prefix" as per the blog you recommended, and it's working nicely. I believe the problems we had last week were all down to the azure issues and not to the balancer configuration, though. Again thanks a lot for the valuable information you gave me. – Vanessa Bullock Nov 28 '14 at 20:10
  • 5
    haproxy 1.7 it's req.cook and res.cook vs .cookie as in the example – Chris DaMour Jan 09 '17 at 04:30
  • 1
    Following @ChrisDaMour 's comments: on HaProxy 1.5 it's also .cook instead of .cookie – Bruno Pérel Oct 17 '17 at 14:29
  • It is unclear to me if the value of the cookie (s1) has to match the internal name of the server (s1) in "server s1 10.0.0.1:80 check cookie s1". Is there a way to prevent server name prediction out of the box? – Chilly Code Apr 02 '21 at 17:02
0

The syntax has changed, for HAProxy version 2.2.9 it should be:

backend mybk
    [...]
    stick-table type string len 64 size 100k expire 15m
    stick store-response res.cook(ASP.NET_SessionId)
    stick on req.cook(ASP.NET_SessionId)
    [...]
    server s1 10.0.0.1:80 check 
    server s2 10.0.0.2:80 check
rconejo
  • 1