I've been given a task of setting up Transparent IP on Nginx in order to pass client address and port to upstream server. The setup look like this:
Client --> Nginx (listens for udp on 90009) --> Service (listens on localhost:59153)
Nginx and Service are set up on the same host.
The basic setup works without any problem - request is caught by nginx, routed to service, the service responds back to nginx, and back to client.
The intended behaviour is to pass request source ip to service, so i setup nginx like this:
stream {
upstream upstream_servers {
server 127.0.0.1:59153;
}
server {
listen 90009 udp;
proxy_bind $remote_addr:$remote_port transparent;
proxy_pass upstream_servers;
}
}
proxy_bind $remote_addr:$remote_port transparent; does its job correctly - ip/port is passed behind nginx to service. However, the service (not surprisingly) is trying to respond directly to given ip/port. The communication with outer world is allowed only on port 90009, so the response is never delivered.
What I tried to do, is to route the response, based on IP Transparency and Direct Server Return with NGINX and NGINX Plus as Transparent Proxy. So I did the following:
iptables -t mangle -N DIVERT
iptables -t mangle -A PREROUTING -p udp -m socket -j DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 1
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p udp -m socket -j TPROXY --tproxy-mark 1 --on-port 90009
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100
But it results in either of two outcomes, depending on how I define the iptables -t mangle -A PREROUTING -p udp -m socket -j TPROXY --tproxy-mark 1 --on-port 90009:
- Income UDP packages are routed, and request never reaches nginx
- None of the packages are routed, response is not delivered
My question is basically, how to route only response through loadbalancer, or, if this approach is wrong, how to attempt to solve this?
