Centos 8, KVM. Similar questions were asked and I tried their answers without success.
I have two public IP addresses IP1 aaa.bbb.ccc.11 and IP2 aaa.bbb.ddd.22 and two network adapters eno1 and eno2. And a lot of guest VMs. aaa.bbb.ccc.1 is the default gateway, and aaa.bbb.ddd.2 is the second gateway for IP2. Both are given by the provider/datacenter (currently being in the same VLAN, but can be changed on request). aaa.bbb.ddd.2 only works using another routing table rt2 (details below).
Port forwarding (iptables qemu hooks) works perfectly for all guest VMs for the public IP aaa.bbb.ccc.11, but I was not able to get it running for aaa.bbb.ddd.22. Let's use the simple case forward port 6000 UDP to the guest (VM2) with IP 192.168.123.2
Site note: I was able to successfully add eno2 and a guest VM to a bridge, had connectivity, but absurdly high UDP package loss. To use a bridge is not the topic (somebody else will work on it).
eno1 and eno2 use the default and similar network-scripts/ifcfg-eno1 and network-scripts/ifcfg-eno2. host 192.168.123.2 lives in the virtual network default1 with virbr2 and gateway 192.168.123.1. They are all standard configs - if they are of help, I can add them here, too. If I add port forwarding from/to my public IP1, it works:
/sbin/iptables -I FORWARD -o virbr2 -p udp -d 192.168.123.2 --dport 6000 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -t nat -I PREROUTING -p udp -d aaa.bbb.ccc.11 --dport 6000 -j DNAT --to 192.168.123.2:6000
# -> nc -u aaa.bbb.ccc.11 6000 works
To add anothor default gateway for aaa.bbb.ddd.22, I needed to add a second routing table and these rules:
ip route add aaa.bbb.ddd.0/24 dev eno2 src aaa.bbb.ddd.22 table rt2
ip route add default via aaa.bbb.ddd.1 dev eno2 table rt2
ip rule add from aaa.bbb.ddd.22/24 table rt2
ip rule add to aaa.bbb.ddd.22/24 table rt2
Now, I can ping aaa.bbb.ddd.22 from the internet, and I can use this IP2 also for SSH (still).
But, the adapted port forwarding has no effect:
/sbin/iptables -I FORWARD -o virbr2 -p udp -d 192.168.123.2 --dport 6000 -m state --state NEW,ESTABLISHED,RELATED -j ACCEPT
/sbin/iptables -t nat -I PREROUTING -p udp -d aaa.bbb.ddd.22 --dport 6000 -j DNAT --to 192.168.123.2:6000
# -> nc -u aaa.bbb.ddd.22 6000 does not work
To list the amount of non-working try-outs I already did would destroy this question. But I can add more information as soon as I know the right direction on how to solve this. I found one solution that I could not try out, a version with MARK and involving "iptables -t mangle -A INPUT -m mac --mac-source $GW2_MAC -j MARK --set-mark 2" . The two gateways aaa.bbb.ddd.1 and aaa.bbb.ddd.2 from the datacenter have the same MAC. The only change the datacenter proposed, was to move aaa.bbb.ccc.0/24 and aaa.bbb.ddd.0/24 to different VLANs, and to leave the VLAN tag, so that I can untag it (I would not know how to do this yet, new topic). So, my question extends, on if and how I can get the port forwading above to work with my second external IP and gateway.
Any help appreciated!
