How should I further troubleshoot port 53 incoming network connections?

Question

I have a Debian 10 Buster server connected to a fiber Internet connection. I'm attempting to run a DNS service on port 53 of the server. What tools or techniques should I use to determine why packets are not arriving at my DNS service on the server?

System setup

One network adapter, eno1 connected to the fiber modem.
One network adapter, eno2 connected to the LAN.
Public IP 162.246.120.21.
nftables as my packet routing subsystem.

nftables rules

table ip filter {                                                                                                                     
        chain input {                                                                                                                 
                type filter hook input priority 0; policy accept;                                                                     
                ct state { established, related } accept
                iif "lo" accept
                tcp dport { ssh, 52, domain, http, https } counter log accept
                udp dport { 52, domain } counter log accept        
                iif "eno2" tcp dport { ssh, domain, http, https, microsoft-ds } counter log accept                                   
                iif "eno2" udp dport { domain, bootps, bootpc } accept
                iif "eno2" ip protocol icmp accept
                counter drop                                      
        }                                                                                                                            
 
        chain output {
                type filter hook output priority 0; policy accept;
                ct state { established, related, new } accept    
                iif "lo" accept                                                                                                      
        }                      
                                                
        chain forward {                                                                                                              
                type filter hook forward priority 0; policy accept;
                iif "eno1" oif "eno2" ct state { established, related } accept
                iif "eno2" oif "eno1" accept
                iif "eno1" oif "eno2" counter drop
                iif "eno1" oif "eno2" counter drop
        }
        chain postrouting {                                                                                                           
                type filter hook postrouting priority 0; policy accept;                                                               
        }                                                                                                                             
                                                                                                                                      
        chain INPUT {                                                                                                                 
                type filter hook input priority 0; policy accept;                                                                     
        }                                               
                               
        chain FORWARD {                                                      
                type filter hook forward priority 0; policy accept;
        }                                                                                                                            
                                                                      
        chain OUTPUT {                            
                type filter hook output priority 0; policy accept;
        }                                                                                                                            
}
table ip nat {                                                                                                                       
        chain postrouting {                                        
                type nat hook postrouting priority 100; policy accept;        
                oifname "eno1" masquerade   
        }                                         
}

Tests

If I run the DNS server on port 52 instead of 53 my tests from remote servers work fine. Testing with dig @162.246.129.21 -p 52 dns.my.tld. Response is fast and correct.
If I run the DNS server on port 53 and test from a system on my LAN it works fine. Testing with dig @162.246.129.21 dns.my.tld.
If I add nft tracing rules I don't see remote traffic in the trace for port 53, but I do for port 52.

chain trace_chain { # handle 39                                                                                                          
   type filter hook prerouting priority -301; policy accept;                                                            
   iif "eno1" tcp dport { 52, 53 } nftrace set 1 # handle 40                                                                    
}

Other

My ISP solemnly swears they don't block any incoming traffic, including port 53.

What other tools or techniques should I try?

Based on what you are saying, it sounds like something is dropping the traffic before it reaches this machine. Is it only 53/udp that is dropped, by the way? — Håkan Lindqvist, Mar 10 '21 at 13:59
As far as I can tell, only 53. I've tested both tcp and udp - results are the same as far as I can tell. — EliRibble, Mar 10 '21 at 15:22
Check your firewall. Check your router's firewall. Don't believe your ISP until it's working. But don't try this on a residential class connection. — Michael Hampton, Mar 10 '21 at 22:11

score 0 · Accepted Answer · edited Apr 10 '23 at 21:15

0

I got my local ISP to finally admit to blocking, and unblock port 53 by running traceroutes to port 53 (both TCP and UDP) from a remote shell and emailing them the results. They stonewalled me for 48 hours prior to that, insisting "no block exists"

edited Apr 10 '23 at 21:15

Dave M

4,514
22
31
30

answered Apr 06 '23 at 23:50

That Guy

16
1

What I ended up doing was created a separate login with sudo privileges and audit logging enabled on the server and gave the login to one of their technicians. They ran essentially the same analysis I did, but this time were convinced that they were in fact blocking incoming UDP port 53. They just didn't realize they were doing it. – EliRibble Apr 19 '23 at 04:06

How should I further troubleshoot port 53 incoming network connections?

System setup

nftables rules

Tests

Other

1 Answers1