0

Having this kind of network layout https://i.stack.imgur.com/NQXDl.jpg I can get it working using mode=switch in with tinc, but not using mode=routed

Those are the network details

On quantum those are the routes/config (hint, external IP has been obfuscated to 1.1.1.1)

netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            1.1.1.1     UGS      vtnet0
8.8.8.8            1.1.1.1     UGHS     vtnet0
10.1.5.0/24        link#2             U        vtnet1
10.1.5.1           link#2             UHS         lo0
10.1.6.0/24        tinc0              US        tinc0
127.0.0.1          link#4             UH          lo0
172.16.0.0/24      link#7             U         tinc0
172.16.0.5         link#7             UHS         lo0
1.1.1.0/29  link#1             U        vtnet0
1.1.1.1     link#1             UHS         lo0


ifconfig vtnet1 | grep inet
    inet 10.1.5.1 netmask 0xffffff00 broadcast 10.1.5.255

ifconfig tinc0 | grep inet
    inet 172.16.0.5 netmask 0xffffff00 broadcast 172.16.0.255

with this tinc host entry

Address=1.1.1.1 655
Subnet=10.1.5.0/24
Cipher=aes-256-cbc
Digest=sha256
-----BEGIN RSA PUBLIC KEY-----
...
-----END RSA PUBLIC KEY-----

on Rocket (hint, external ip has been obfuscated to 2.2.2.2)

netstat -rn4
Routing tables

Internet:
Destination        Gateway            Flags     Netif Expire
default            2.2.2.2      UGS      vtnet0
8.8.8.8            2.2.2.2      UGHS     vtnet0
10.1.5.0/24        tinc0              US        tinc0
10.1.6.0/24        link#2             U        vtnet1
10.1.6.1           link#2             UHS         lo0
2.2.2.0/29   link#1             U        vtnet0
2.2.2.2      link#1             UHS         lo0
127.0.0.1          link#4             UH          lo0
172.16.0.0/24      link#7             U         tinc0
172.16.0.6         link#7             UHS         lo0

ifconfig vtnet1 | grep inet
    inet 10.1.6.1 netmask 0xffffff00 broadcast 10.1.6.255

ifconfig tinc0 | grep inet
    inet 172.16.0.6 netmask 0xffffff00 broadcast 172.16.0.255

with this tinc host entry

Address=2.2.2.2 655
Subnet=10.1.6.0/24
Cipher=aes-256-cbc
Digest=sha256
-----BEGIN RSA PUBLIC KEY-----
...
-----END RSA PUBLIC KEY-----

Further details

When doing a ping -S 10.1.5.1 10.1.6.1 from quantum to rocket, I see on the FW that the ICMP package passes the firewall, but it never reaches rocket, no block or pass.

Question

What is wrong with my setup, why is mode=routed not working?

Eugen Mayer
  • 327
  • 1
  • 5
  • 15

1 Answers1

1

After wrapping my head around this for a time, i found 2 issues, which stacked, held me back from getting to the solution earlier:

1. This is actually a opnsense thing

This is what i had / you will need: For both

Quantum

  • You need to create a Gateway Rocket, which is configured to go through tinc0 with "dynamic" ( do not enter an IP on Gateway field, see problem below and UPDATE )
  • You need a Route to 10.1.6.0/24 through TINCGW
  • Add your tinc0 interface in the Interface section, name it TINCLAN. You can configure a ipv4 address or you don't, does not matter. If you do, use your tinc-up configured address. Doing this enabled you to create FW Rules for TINCLAN - which we will need. Hint: the interface has been created as tinc0 and not tun0 since you otherwise could not add it in opnsense, since all tun* are filtered out ( legacy bug )
  • A FW RULE to allow TINCLAN 2 TINCLAN
  • A FW RULE to allo TINCLAN 2 LAN ( Lan is 10.1.5.0/24 )

Rocket

  • You need to create a Gateway Rocket, which is configured to go through tinc0 with "dynamic" ( do not enter an IP on Gateway field, se problem below and UPDATE )
  • You need a Route to 10.1.5.0/24 through TINCGW
  • Add your tinc0 interface in the Interface section, name it TINCLAN. See quantum above
  • A FW RULE "dest TINCLAN" on the TINCLAN tab to allow TINCLAN 2 TINCLAN
  • A FW RULE "dest LAN" on the TINCLAN tab to allow TINCLAN 2 LAN ( Lan is 10.1.6.0/24 )

Problem with this / fooled me: When you restart tincd, the tinc0 interface loses its IP and the route is being removed automatically, starting tincd will not readd those. Either you need opnsense to force to reapply them, restart the server or most probably, but those routes into tinc-up. Yet finding a perfect solution for this I did not finish with, but maybe this is also out of scope for this question.

When you started tincd and set the Route after that, things should work.

2. Typo, not important (stupid second issue, but taught me a lesson)

I actually put Subnet 10.0.6.0/24 not 10.1.6.0/24 in the rocket hosts/rocket file - that was blocking the packages on tinc-level

UPDATE 1

Finally I also solved the glitch / obvious setup issue for 1. What you need is

  • When creating the Gateway TINCGW, do not enter a Gateway IP in the "Gateway" field - so its dynamic. This way the route is created to be10.1.6.0/24 tinc0` rather then on the ip 172.16.0.5 .. this way, when you end tincd and tinc0 looses its ip, that route is not removed, since tinc0 is still there. So this then survives tincd restarts or similar.

I will merge this into my answer later, to have people understand the issue / workflow.

Eugen Mayer
  • 327
  • 1
  • 5
  • 15