0

I am running haproxy 1.5.18-3 on CentOS 7 and need to use the stick-table feature to make sure traffic for a specific user persists to a given server.

Things work fine when connections come in slowly, however when there's numerous simultaneous connections and a stick-table entry doesn't exist yet some requests will be sent to both backend servers until they eventually stick to just one.

For example, using Apache Bench like so:

ab -c 10 -n 30 'http://example.com/index.php?email=abc@example.com'

I see this in the haproxy log:

May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50812 [08/May/2017:14:49:10.934] http_front backend/server-1 0/0/0/7/7 200 222 - - ---- 9/9/9/4/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50811 [08/May/2017:14:49:10.933] http_front backend/server-2 0/0/0/8/8 200 222 - - ---- 8/8/8/4/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50816 [08/May/2017:14:49:10.935] http_front backend/server-1 0/0/0/7/7 200 222 - - ---- 7/7/7/1/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50819 [08/May/2017:14:49:10.935] http_front backend/server-2 0/0/1/6/7 200 222 - - ---- 6/6/6/1/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50814 [08/May/2017:14:49:10.935] http_front backend/server-1 0/0/0/7/7 200 222 - - ---- 5/5/5/1/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50810 [08/May/2017:14:49:10.933] http_front backend/server-1 0/0/0/9/9 200 222 - - ---- 4/4/4/0/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50813 [08/May/2017:14:49:10.934] http_front backend/server-2 0/0/0/8/8 200 222 - - ---- 3/3/3/0/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50815 [08/May/2017:14:49:10.935] http_front backend/server-2 0/0/0/7/7 200 222 - - ---- 2/2/2/0/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50817 [08/May/2017:14:49:10.935] http_front backend/server-2 0/0/0/7/8 200 222 - - ---- 1/1/1/0/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50818 [08/May/2017:14:49:10.935] http_front backend/server-1 0/0/1/6/8 200 222 - - ---- 0/0/0/0/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50820 [08/May/2017:14:49:10.967] http_front backend/server-1 0/0/0/5/5 200 222 - - ---- 3/3/2/2/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50821 [08/May/2017:14:49:10.968] http_front backend/server-1 0/0/0/4/4 200 222 - - ---- 2/2/2/2/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50823 [08/May/2017:14:49:10.972] http_front backend/server-1 0/0/1/5/6 200 222 - - ---- 7/7/7/7/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
May  8 14:49:10 localhost haproxy[4996]: 1.2.3.4:50822 [08/May/2017:14:49:10.972] http_front backend/server-1 0/0/0/8/8 200 222 - - ---- 6/6/6/6/0 0/0 "GET /index.php?email=abc@example.com HTTP/1.0"
[...]

After this point haproxy correctly sends all traffic to server-1. When the stick-table entry expires the problem occurs again.

I have tried persisting off a url parameter and source address, both exhibit the same issue.

Is haproxy unable to properly handle numerous simultaneous (concurrent) requests like this? Is there something I can do to get this to work as desired?

Thanks.

Gene
  • 3,663
  • 20
  • 39

1 Answers1

0

What's happening is that before HAProxy creates a stick-table entry it needs validation that the first connection succeeded before said entry is created. With so many requests coming in so close to each other it hasn't had time to validate that it worked, so it will balance the connections across the backend servers until then.

Is there something I can do to get this to work as desired?

I ended up using the url_param balancing algorithm and set it to use the email parameter. 

  balance url_param email

HAProxy takes the value of the url_param, hashes it, and determines what backend server to send it to and always sends it there, at least until there's a change in the backend servers (e.g. adding a node). At this point stick-tables can be used to help avoid traffic being misrouted when the backend changes.

Gene
  • 3,663
  • 20
  • 39