2

So I've been pulling my hair out for the last 24 hours to sort this out.

Long-story-short, my home internet runs over a 4G Mobile network, so this means I cannot Port Forward through my main Router. I have a few services internaly that I want to make available like a HTTP Server, RD Gateway, etc.. All kinds of stuff.

Diagram enter image description here Ultimately, my end goal is to utilize a VPS I have in the cloud to forward ports to services inside my home network. I have an Ubuntu VPS Cloud Server with a public IP (lets just say its 111.111.111.111 for now...) running WireGuard connecting to another Ubuntu Box running in a VM inside my home network. I have managed to get the WireGuard connection up and running. I can ping both my VPS and my internal Ubuntu VM over the WireGuard tunnel. Just to be clear, I don't want to forward my home network's internet traffic through the WireGuard VPN; I only want to forward requests hitting my VPS on specific ports going to specific devices on my home network, and then my home network services being able to reply back to it.

Let's say I have three different services inside my home network that I want to access

  • HTTP Server - Located inside my home network at 192.168.1.252. This will be accepting HTTP traffic on port 80
  • Another HTTP Server - This time it is located directly on the Ubuntu VM inside my home network. This is accessible from my internal network on the IP 192.168.1.247. Because of my WireGuard, this is already accessible from my VPS server when my VPS connects to 192.168.4.4:80. It is currently not accessible from the outside world. I also understand that we are looking at an Port Conflict here as my other HTTP server is also listening on port 80. The main aim here is that when port 8080 is hit on the VPS, I want it to hit this HTTP Server. If this makes it all complicated, then I don't mind scrapping this HTTP Server and just keeping the first HTTP Server.
  • RDP Server - This will be running on 192.168.1.251 over port 3389. I already know what you are thinking regarding a 3389 redirect, but this is just for example purposes. In the end I'll be sorting out a RD Gateway Server through port 443. I kinda wanted to use this example, and then mold it to an RD Gateway Server further down the line. I just want to get a 3389 RDP up and running for now.

I have tried every single iptables command under the sun to get this up and running, to no success. I have scanned the web everywhere and I cannot find my answer anywhere. I've managed to gather quite a lengthy list of PostUp and PostDown rules inside my wg0.conf file on my VPS (all of which I have commented out btw...). I understand that I'll also have to create similar PostUp and PostDown iptables commands on my internal Ubuntu VM to get this to work.

The closest thing that managed to get this to work was using a Reverse Proxy Program called Caddy. Even though I managed to get my internal Ubuntu HTTP server accessible from the internet, I couldn't get it to connect to my other HTTP Server and my RDP box.

Just as a heads up, I am not brilliant at Linux. I know how to get around some parts of the place, but it involves Googleing every single step of the way. I'm kinda suprised that I managed to get this far.

Just as a heads up, I do already have net.ipv4.ip_forward=1 set in my /etc/sysctl.conf file on my VPS

Any help will be greatly appreciated with how I am supposed to set up this port forwarding system!

Mircea Vutcovici
  • 17,619
  • 4
  • 56
  • 83
Flakie
  • 31
  • 3
  • I would test with `tcpdump -ni port 80` to see if I can see the packets on the path of each hop, to isolate the problem. With `iptables -L -nv` you can see how many times a firewall rule was matched - this is also good for debugging as it tells you if the rule was reached and when (if you run multiple times). – Mircea Vutcovici Aug 05 '23 at 15:44

2 Answers2

1

After many, many hours. I have managed to resolve this with help from Mircea pointing me in the right direction.

Only issue is I really wish I fully understood what I just did, but I'll have a crack at it anyway and post up what I did.

Turns out it was an iptables issue. Not only did I have to provide PREROUTING rules, but I also had to set up the FORWARD rules. This was done by creating FORWARD rules for each incoming port, along with setting up the interfaces to allow FORWARD rules.

After that, I then set up the PREROUTING rules as Mircea had pointed out.

Finally, I had to set up POSTROUTING rules.

Just as a heads up for anyone that is wanting to follow along, in my example above, I said that my main HTTP server lived on 192.168.1.251. In the code below, I actually set this to a different HTTP server on 192.168.1.190 (just for testing). Same thing for the RDP server. That was described as 192.168.1.252, but I actually connected it to a different RDP server at 192.168.1.241

Here is a snippet of my wg0.conf file that does all my iptable rules from the Cloud VPS side

[Interface]
Address = 192.168.4.1/32
ListenPort = xxxx
PrivateKey = xxxx

PostUp = iptables -P FORWARD DROP

##FORWARD Port 80, 8080, 3389
PostUp = iptables -A FORWARD -i ens192 -o wg0 -p tcp --syn --dport 80 -m conntrack --ctstate NEW -j ACCEPT
PostUp = iptables -A FORWARD -i ens192 -o wg0 -p tcp --syn --dport 8080 -m conntrack --ctstate NEW -j ACCEPT
PostUp = iptables -A FORWARD -i ens192 -o wg0 -p tcp --syn --dport 3389 -m conntrack --ctstate NEW -j ACCEPT

##Generic Forwards
PostUp = iptables -A FORWARD -i ens192 -o wg0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
PostUp = iptables -A FORWARD -i wg0 -o ens192 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

##Drop FORWARD Port 80, 8080, 3389
PostDown = iptables -D FORWARD -i ens192 -o wg0 -p tcp --syn --dport 80 -m conntrack --ctstate NEW -j ACCEPT
PostDown = iptables -D FORWARD -i ens192 -o wg0 -p tcp --syn --dport 8080 -m conntrack --ctstate NEW -j ACCEPT
PostDown = iptables -D FORWARD -i ens192 -o wg0 -p tcp --syn --dport 3389 -m conntrack --ctstate NEW -j ACCEPT

##Drop Generic Forwards
PostDown = iptables -D FORWARD -i ens192 -o wg0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
PostDown = iptables -D FORWARD -i wg0 -o ens192 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT



##Port 80
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 192.168.4.4:80
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 80 -j DNAT --to-destination 192.168.4.4:80

PostUp = iptables -t nat -A POSTROUTING -o wg0 -p tcp --dport 80 -d 192.168.4.4 -j SNAT --to-source 192.168.4.1
PostDown = iptables -t nat -A POSTROUTING -o wg0 -p tcp --dport 80 -d 192.168.4.4 -j SNAT --to-source 192.168.4.1


##Port 8080
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.4.4:8080
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.4.4:8080

PostUp = iptables -t nat -A POSTROUTING -o wg0 -p tcp --dport 8080 -d 192.168.4.4 -j SNAT --to-source 192.168.4.1
PostDown = iptables -t nat -A POSTROUTING -o wg0 -p tcp --dport 8080 -d 192.168.4.4 -j SNAT --to-source 192.168.4.1


##Port 3389
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 3389 -j DNAT --to-destination 192.168.4.4:3389
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 3389 -j DNAT --to-destination 192.168.4.4:3389

PostUp = iptables -t nat -A POSTROUTING -o wg0 -p tcp --dport 3389 -d 192.168.4.4 -j SNAT --to-source 192.168.4.1
PostDown = iptables -t nat -A POSTROUTING -o wg0 -p tcp --dport 3389 -d 192.168.4.4 -j SNAT --to-source 192.168.4.1


[Peer]
PublicKey = xxxxx
AllowedIPs = 192.168.4.4/32

As for my Internal WireGuard VM, here is the wg0.conf file for that

[Interface]
PrivateKey = xxxx
Address = 192.168.4.4/32

PostUp = iptables -P FORWARD DROP

##FOWARD 8080, 3389
PostUp = iptables -A FORWARD -i wg0 -o ens34 -p tcp --syn --dport 80 -m conntrack --ctstate NEW -j ACCEPT
PostUp = iptables -A FORWARD -i wg0 -o ens34 -p tcp --syn --dport 3389 -m conntrack --ctstate NEW -j ACCEPT

#Generic Forwards - Setting up Interfaces for Forwarding
PostUp = iptables -A FORWARD -i wg0 -o ens34 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
PostUp = iptables -A FORWARD -i ens34 -o wg0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

#Drop FORWARD 8080, 3389
PostDown = iptables -D FORWARD -i wg0 -o ens34 -p tcp --syn --dport 80 -m conntrack --ctstate NEW -j ACCEPT
PostDown = iptables -D FORWARD -i wg0 -o ens34 -p tcp --syn --dport 3389 -m conntrack --ctstate NEW -j ACCEPT

#Drop Generic Forwards
PostDown = iptables -D FORWARD -i wg0 -o ens34 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
PostDown = iptables -D FORWARD -i ens34 -o wg0 -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT



#Port 8080 Inbound to external 80
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.1.190:80
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 8080 -j DNAT --to-destination 192.168.1.190:80

PostUp = iptables -t nat -A POSTROUTING -o ens34 -p tcp --dport 80 -d 192.168.1.190 -j SNAT --to-source 192.168.1.247
PostDown = iptables -t nat -A POSTROUTING -o ens34 -p tcp --dport 80 -d 192.168.1.190 -j SNAT --to-source 192.168.1.247


#Port 3389
PostUp = iptables -t nat -A PREROUTING -p tcp --dport 3389 -j DNAT --to-destination 192.168.1.241:3389
PostDown = iptables -t nat -D PREROUTING -p tcp --dport 3389 -j DNAT --to-destination 192.168.1.241:3389

PostUp = iptables -t nat -A POSTROUTING -o ens34 -p tcp --dport 3389 -d 192.168.1.241 -j SNAT --to-source 192.168.1.247
PostDown = iptables -t nat -A POSTROUTING -o ens34 -p tcp --dport 3389 -d 192.168.1.241 -j SNAT --to-source 192.168.1.247



[Peer]
PublicKey = xxxx
AllowedIPs = 192.168.4.1/32
Endpoint = xxxx
PersistentKeepalive = 25

I hope this helps anyone else with a similar issue!

Flakie
  • 31
  • 3
0

You need to use a DNAT rule on INPUT table on the Cloud VM.

iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 192.168.4.4:80

On Wireguard VM you might need to use also a DNAT rule on INPUT table, because I do not expect to have the Wireguard prefix route to be present in the routing table of all your computers in the home LAN. At this moment you have 2 routers, but no dynamic routing. This means the easiest way is to use DNAT on wireguard.

iptables -t nat -A PREROUTING -p tcp --dport 80 -j DNAT --to-destination 192.168.1.252:80
Mircea Vutcovici
  • 17,619
  • 4
  • 56
  • 83
  • Thanks for getting back to me! I placed the above iptables commands into my wg0.conf file (one for PostUp, and then a -D for PostDown), however it's still not accessing my internal WireGuard VM. I did a TCPDump on both boxes to see whats happening, and my Cloud VM is picking up the request and keeps writing the same line of {My Phones Public IP} > KERN-VPS.http. The Internal WireGuard VM shows nothing on TCPDump when this happens. At the moment, I am just trying to access my WireGuard VM HTTP Server from the internet, so I haven't put anything in the iptables of my Internal VM – Flakie Aug 05 '23 at 15:59
  • Check if the rules is matched. "-A" means append, but this will append at the end of all rules. If you have any other rule that is matched before, the appended rule will be ignored. – Mircea Vutcovici Aug 05 '23 at 16:07
  • So a bit more Googleing, and I've worked out that my only rule sitting in the PREROUTING Chain is the one that you stated above. Each time I'm making a request the packets for the rule is going up, so I can only assuming that rule is being hit. Regardless though, still nothing appearing in the TCPDump on the internal VM side – Flakie Aug 05 '23 at 17:44
  • On which interface are you listening with tcpdump? Is it the wireguard one, with IP 192.168.4.4 ? – Mircea Vutcovici Aug 05 '23 at 19:33
  • I ran tcpdump with it listening to any interface on port 80 -- tcpdump -ni any port 80. What should be noted is that if I run a curl 192.168.4.4 from the cloud VM, it does appear in my TCPDump on the Internal VM – Flakie Aug 05 '23 at 20:04
  • Just as a heads up with the TCPDump, when I run it on my Cloud VPS, it repeats the same line over and over again ({My Phones Public IP} > KERN-VPS.http. etc...). I'm assuming on what should be happening is that there should be another line below it something along the lines of 192.168.4.1 >192.168.4.4, but this is not appearing – Flakie Aug 05 '23 at 20:23
  • If you can see it on TCPDump on the Internal VM (I assume this is the Wireguard VM), this means that the first DNAT rule is ok. Check if the rule is matched on the Wireguard VM. Check if the packet exits the Wireguard VM. Using tcpdump on all interfaces is not recommended for this troubleshooting. You need to see of the packet enters and exits WireguardVM, And that the destination IPs are changed. – Mircea Vutcovici Aug 06 '23 at 12:57