Using Squid and/or iptables to share ip address over a bridge

Question

Edited for additional clarity and added links to other attempted solutions.

I have been attempting this for several days now with one other developer, and we are getting nowhere and there are a number of comments on-line about how there are no examples to do this sort of thing (including someone who wrote some c code to do something similar though not exactly this). We have attempted to implement the solution described on SuperUser as well, but so far it does not seem like the local http server receives any of the requests as expected.

What we are trying to do:

On a device (test device) that sits between another device (mini computer) and the network. We want the test device to use the ip address of the mini computer to communicate with the control server -- in other words, we don't want it to have to have its own IP address but use that of the minicomputer for control commands (e.g., block network traffic, resume network traffic). Things are set up like so:

Mini Computer|    |            Test Device           |    | LAN
Ethernet     |<-->|eth_minicomp<-->br0<-->eth_network|<-->| Ethernet

So for traffic that is:

• coming from the control IP address, AND
• destined for the mini computer IP address

We want the test device to intercept (and NOT forward), but use locally.

Whereas for traffic that is:

• comping from the test device, AND
• destined for the control IP address

We want it going out the eth_network interface with the src address being the mini computer ip address.

Latest Attempt

I have a device set up as a transparent bridge which works:

# Bring interfaces down
ip link set dev eth_minicomp down
ip link set dev eth_network down
# Create bridge
ip link add name br0 type bridge
ip link set dev br0 up
# Remove IP addresses from interfaces
ip address flush dev eth_minicomp
ip address add 0.0.0.0 dev eth_minicomp
ip address flush dev eth_network
ip address add 0.0.0.0 dev eth_network
# Bring interfaces back up
ip link set dev eth_minicomp up
ip link set dev eth_network up
# Set promisc (not sure about on br0, but should not have an effect)
ip link set dev eth_minicomp promisc on
ip link set dev eth_network promisc on
ip link set dev br0 promisc on
# Add interfaces to bridge
ip link set dev eth_minicomp master br0
ip link set dev eth_network master br0

I had been hoping to use iptables/tproxy or perhaps Squid to handle this by routing the desired TCP/IP traffice to lo (127.0.0.1), but cannot seem to get this to work. My latest attempt was trying to use

sysctl net.ipv4.ip_forward=1
sysctl net.ipv4.conf.lo.rp_filter=1
iptables -t mangle -F
iptables -t mangle -X
iptables -t mangle -N DIVERT
iptables -t mangle -A DIVERT -j MARK --set-mark 0x01/0x01
iptables -t mangle -A DIVERT -j ACCEPT
iptables -t mangle -A PREROUTING -p tcp -m socket -j DIVERT
iptables -t mangle -A PREROUTING -s $CONTROLLER_IP -p tcp -j TPROXY \
    --tproxy-mark 0x1/0x1 --on-port 80
ip route flush table 100
ip rule add fwmark 1 lookup 100
ip route add local 0.0.0.0/0 dev lo table 100

TPROXY seem to require at least the net.ipv4.ip_forward set 1,2, however, following the procedure on the Squid TPROXY Feature page does not seem to be set up for this type of solution.

And various permutations on -s, -d, --on-port, etc. It seems that I could use the Suid man in the middle setup to do something like this, but I do not see how. Trying to search for Suid man in the middle or Squid localhost proxy on SO returns a lot of not-quite-what-i'm-looking-for questions.

So how do we route these packets to a local server on the test device for handling? RTFM responses are more than welcome, we just cant find the fabulous manual.

Answer 1

Got it working with help from a team member using ebtables and iptables.

The biggest surprise in getting this working was finding out that if you use ebtables to create an Ethernet bridge, you have to DROP the Ethernet frames in order for them to get kicked up to the network layer. We all thought that DROP actually dropped the Ethernet frame and therefore the TCP/IP packets. Go figure.

We now have a device that can share the MAC and IP address of the computer to which it is attached and still communicate without disrupting the computer.

INT_IP=169.254.1.1
SRC_IP=192.168.1.2
DST_IP=192.168.1.3
EXT_PORT=80
INT_PORT=54321

# Bring interfaces to bridge down
ip link set dev eth1 down
ip link set dev eth2 down
# Remove any ip addresses on the interfaces
ip address flush dev eth1
ip address flush dev eth2
ip address add 0.0.0.0 dev eth1
ip address add 0.0.0.0 dev eth2
# Bring interfaces back up
ip link set dev eth1 up
ip link set dev eth2 up
# Set promiscuous on the interfaces
ip link set dev eth1 promisc on
ip link set dev eth2 promisc on
# Create bridge
ip link add name br0 type bridge
ip link set dev br0 up
# Add interfaces to bridge
ip link set dev eth1 master br0
ip link set dev eth2 master br0
# Add a local private IP to the bridge
ip address add $INT_IP dev "br0"
# Allow forwarding
sysctl -w net.ipv4.ip_forward=1
# Set up ethernet bridge with ebtables.
# NOTE the drop. Completely counterintuitive.

ebtables -t broute -A BROUTING -p IPv4 --ip-source $SRC_IP \
     --ip-destination $DST_IP --ip-proto tcp --ip-dport \
     $EXT_PORT -j redirect --redirect-target DROP
ebtables -t broute -A BROUTING -p IPv4 --ip-proto tcp \
     --ip-sport $INT_PORT -j redirect --redirect-target \
     DROP

# Set up iptables to handle diverting requests that originate
# from $SRC_IP destined for $DST_IP on port $EXT_PORT and send
# them to $INT_IP and $EXT_PORT in stead where you can have a
# service / thingy to handle them.
iptables -t nat -A PREROUTING -p tcp -s $SRC_IP -d $DST_IP \
     --dport $EXT_PORT -j DNAT \
     --to-destination $INT_IP:$INT_PORT
iptables -t nat -A POSTROUTING -p tcp -d $INT_IP \
     --dport $EXT_PORT -j SNAT --to-source \
     $DST_IP:$EXT_PORT
iptables -t nat -A POSTROUTING -j MASQUERADE

Now if you try to reach $DST_IP on port $EXT_PORT from $SRC_IP, it will be routed to $INT_IP on $INT_PORT in stead. Conversely, if you try to send data to $INT_IP on $INT_PORT from the system on which you configured this, all traffic will go to $SRC_IP on $EXT_PORT

-2 karma! Woohoo!