I have two servers, .105 and .104, and I am trying to troubleshoot 'no route to host' issues when .105 tries to connect to .104 over ssh or tcp.
.104 has a service running on port 16000. .105 used to connect to .104 via port 16000 but .105 had to be rebuilt and is now unable to connect to .104 now over that port. .105 can still ping .104. Both run CentOS 8. Firewall port port 16000 is still open on .104. Another server, .103, is able to connect to .104 via port 16000. All servers are on the same LAN.
On .105:
(base) [root@web etc]# nc 192.168.0.104 16000
Ncat: No route to host.
(base) [root@web etc]# ssh 192.168.0.104
ssh: connect to host 192.168.0.104 port 22: No route to host
.105 also cannot ssh to .104 ('no route to host') but can ssh to .103. .103 can ssh into .104. .104 can ssh into both servers. I tried hostname and ip address.
On .105:
base) [root@web etc]# nmap --reason -p 16000 192.168.0.104
Starting Nmap 7.70 ( https://nmap.org ) at 2020-01-20 23:23 UTC
Nmap scan report for 192.168.0.104
Host is up, received arp-response (0.000097s latency).
PORT STATE SERVICE REASON
16000/tcp filtered unknown admin-prohibited ttl 64
MAC Address: 00:50:56:A5:F5:47 (VMware)
Nmap done: 1 IP address (1 host up) scanned in 0.51 seconds
Not sure where the filtering would occur as there's nothing between the two servers. Both are vm's on the same physical box.
I've double and triple checked network and firewall configurations but am at a loss of what the problem could be. Any help would be appreciated.
On .104 (server .105 is trying to connect to):
[root@winst ]# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: ens192
sources:
services: cockpit dhcpv6-client http ssh
ports: 16000/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
On .103 (third server just for troubleshooting, can connect to .104):
[root@monitoring ]# nmap 192.168.0.104
Starting Nmap 7.70 ( https://nmap.org ) at 2020-01-20 23:53 UTC
Nmap scan report for 192.168.0.104
Host is up (0.000097s latency).
Not shown: 996 filtered ports
PORT STATE SERVICE
22/tcp open ssh
80/tcp closed http
3306/tcp open mysql
9090/tcp open zeus-admin
MAC Address: 00:50:56:A5:F5:47 (VMware)
Nmap done: 1 IP address (1 host up) scanned in 14.84 seconds
On .105 (the problem server):
Starting Nmap 7.70 ( https://nmap.org ) at 2020-01-20 23:53 UTC
Nmap scan report for 192.168.0.104
Host is up (0.000078s latency).
Not shown: 999 filtered ports
PORT STATE SERVICE
2049/tcp open nfs
MAC Address: 00:50:56:A5:F5:47 (VMware)
Nmap done: 1 IP address (1 host up) scanned in 5.63 seconds
Could it have something to do with a new zone I created for nfs? I may have added nfs zone not too long before I had to rebuild .105 and may not have noticed that it broke other access.
On .104:
[root@winst]# firewall-cmd --get-active-zones
nfs
sources: 192.168.0.105 192.168.0.5
public
interfaces: ens192
