4

My requirement is to have ~50 haproxy http backends, which will be added and removed dynamically (without my involvement) anywhere on the internal network. I can assume that each http backend knows its IP address and knows the IP address of haproxy. I can assume that I can add a little bit of code to each http backend, to allow it to register with haproxy whenever it spins up and deregister before it spins down.

I have haproxy listening on port 80:

frontend main *:80

Additionally I expose a stat socket:

stats socket 0.0.0.0:8080

My setup works & in particular I can connect to the stats socket using socat readline TCP4:<haproxyIP>:8080

Let us assume that we have two http backends serving on 10.0.0.1:4040 and 10.0.0.2:4040, and that haproxy can reach both addresses.

What stat socket command do I need to issue over the stat socket so that any request to <haproxyIP>:8080/backend1/status is routed to 10.0.0.1:4040/status and any request to <haproxyIP>:8080/backend2/version is routed to 10.0.0.2:4040/version?

In general: how do I dynamically proxy http requests from [GET|POST] <haproxyIP>:<haproxyPort>/<backendID>/<remainingPath> to [GET|POST] <backendIP>:<backendPort>/<remainingPath> given that I know a map from backendID -> backendIP, backendPort.

Adam Kurkiewicz
  • 141
  • 1
  • 4

2 Answers2

2

If all of your requests arriving to this frontend are in such form:

frontend main
    bind *:80 # Use separate bind directives as they are supported by new HAProxy versions
    use_backend %[path,fields(2,/)]

backend backend1
    reqrep ^([^\ :]*)\ /backend1/(.*)     \1\ /\2
    server foo ...

backend backend2
    reqrep ^([^\ :]*)\ /backend2/(.*)     \1\ /\2
    server bar ...

If you have some requests that are not in this format, you can add more use_backend-s or a default_backend. You can also have the orchestration tool that generates the config generate a flat file with all the backend names and you can then use it for even more logic

acl existing_backends path -f all_backends.txt -m beg

Where the all_backends.txt would look like:

/backend1/
/backend2/
...

Depending on the exact HAProxy version you are using, this could be somewhat optimzied (using quotes, variables, etc.), but this way it should work with 1.5+.

nmerdan
  • 51
  • 4
  • Looks neat. I've ended up writing a new nginx file dynamically every 30 seconds (and restarting nginx after that period). But I guess this would work too. Will there be a need to restart HAProxy every time after changing the config? – Adam Kurkiewicz Jun 12 '17 at 08:14
  • Yes, you need too reload (not restart) HAProxy every time you add a backend. The reload won't break existing connections and will launch a new process. The old one will finish once all connections are served. You can read more about it [here](https://www.haproxy.com/blog/truly-seamless-reloads-with-haproxy-no-more-hacks/). – nmerdan Jun 12 '17 at 10:15
1

you can achieve that using the unix socket commands. Take a look at point 9.2 of the management.txt doc file.

There are third party tools that use this 'api' like haproxyctl, but I do not have any experience with them.

natxo asenjo
  • 5,739
  • 2
  • 26
  • 27
  • 1
    I know that it can be done using this API -- my question is about **how** to do it. I've read the docs, but it didn't make me any wiser. – Adam Kurkiewicz Jun 28 '16 at 11:10
  • you specified the http api, I referred to the unix sockets, quite a difference. – natxo asenjo Jun 28 '16 at 11:43
  • 1
    @nataxoasenjo, I don't think there's any difference: it's the same API, which is called "stats socket". There are 2 different ways to expose this API, one way is indeed UNIX socket, but this is not applicable to my scenario, since the requests to this API will be coming from intranet instead of locally. Another way is to expose it via a tcp socket (which I've done). I didn't expose anything through HTTP API. AFAIK haproxy doesn't have a HTTP API unless you [pay for it](https://www.haproxy.com/doc/aloha/7.0/api/webbased.html), which I'm not inclined to. – Adam Kurkiewicz Jun 28 '16 at 11:54
  • 1
    you are right, it is the same api, i stand corrected. I cannot help you with the exact commands, but I would use socat and start issuing commands (it even has a primitive help function) until you get what you want and use those same commands in your tcp socket interface – natxo asenjo Jun 28 '16 at 12:12