3

I have a scenario where I need to route TCP traffic to a dynamic set of backend servers (Kubernetes pods to be exact but we can ignore that for purposes of this post) through a proxy like HAProxy or nginx. The traffic needs to be routed based on a key (call it the routing_key) provided by the client in the TCP payload.

I see that both nginx and HAProxy support consistent hashing. However, from what I can tell based on HAProxy's manual (see "balance" section), there's no way to perform consistent hashing based on a TCP payload. Payload-based load balancing seems to be limited to L7 HTTP parameters like Header and URI params. This post outlines a method for statically balancing based on string matching a TCP payload, but my case is more dynamic so a true consistent hashing approach is much preferred.

Nginx appears to offer a bit more flexibility in that you can set the hashing value to an arbitrary variable as shown here. This appears to work for both L7 (the "backend" stanza) and L4 (the "stream" stanza). However, I'm a bit hazy on what you are and aren't allowed to do for variables. Does anyone have an example of setting a variable to be a value extracted from the TCP payload and using that for consistent hashing?

Final bonus question: the routing_key value is actually an AES-GCM encrypted value. The proxy server would have access to the key used to decrypt this value. Is it possible to have nginx grab the routing key value from the TCP payload, decrypt it using the known key, and then use that for consistent hashing? Would that involve creating an nginscript module?

Lee Hampton
  • 410
  • 5
  • 13

1 Answers1

3

In the HAProxy 2.1 can you use aes_gcm_dec(...) in combination with req.payload(...) for such a requirement.

My idea, untested.

listen tcp-in
  bind :443 ssl cert 
  tcp-request inspect-delay 10s
  tcp-request session set-var(sess.routingkey) req.payload(0,500)

  # for consistent hashing try this
  hash-type consistent wt6

  use_backend %[var(sess.routingkey),aes_gcm_dec(128,txn.nonce,Zm9vb2Zvb29mb29wZm9vbw==,txn.aead_tag)]

Here are also the links to the html Documentation.
aes_gcm_dec
req.payload
hash-type consistent ... is described at hash-type

Aleksandar
  • 2,442
  • 3
  • 15
  • 24
  • This might just be my ignorance of how HA Proxy works, but does this satisfy the requirement of consistent hashing? I'm not quite seeing how the decrypted payload gets converted to a hash that will consistently map to a particular backend out of many. – Lee Hampton May 20 '20 at 00:47
  • you can try to set `hash-type consistent wt6` http://cbonte.github.io/haproxy-dconv/2.1/configuration.html#4.2-hash-type . But you are right I also don't see a option to balance based on tcp payload. maybe you can ask on the mailing list for some ideas how to solve your issue haproxy@formilux.org – Aleksandar May 20 '20 at 08:19