0

I have two servers running RHEL 5.8. SRV1 (10.58.192.36) has a service running on port 9002 and I need to connect to this service from SRV2 (10.58.192.42).

[root@SRV1 ~]# netstat -anp | grep 9002
tcp 0 0 :::9002 :::* LISTEN 20740/java

If I try to connect to this service from SRV1 itself, it works.

[root@SRV1 ~]# telnet 10.58.192.36 9002
Trying 10.58.192.36...
Connected to SRV1 (10.58.192.36).
Escape character is '^]'.
SheerMachineInterface

But if I try to connect to this service from SRV2 using the same command, it gets disconnected immediately.

[root@SRV2 ~]# telnet 10.58.192.36 9002
Trying 10.58.192.36...
Connected to SRV1 (10.58.192.36).
Escape character is '^]'.
Connection closed by foreign host.
You have new mail in /var/spool/mail/root
[root@SRV2 ~]#

I have already tried disabling both iptables and ip6tables services on SRV1 but no luck. This is the current iptables and ip6tables configuration.

[root@SRV1 ~]# iptables -n --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
ACCEPT     tcp  --  0.0.0.0/0            0.0.0.0/0           tcp dpt:9002 

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         
[root@SRV1 ~]#
[root@SRV1 ~]#
[root@SRV1 ~]# ip6tables -n --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         
RH-Firewall-1-INPUT  all      ::/0                 ::/0               

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         
RH-Firewall-1-INPUT  all      ::/0                 ::/0               

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination         

Chain RH-Firewall-1-INPUT (2 references)
target     prot opt source               destination         
ACCEPT     all      ::/0                 ::/0               
ACCEPT     icmpv6    ::/0                 ::/0               
ACCEPT     esp      ::/0                 ::/0               
ACCEPT     ah       ::/0                 ::/0               
ACCEPT     udp      ::/0                 ff02::fb/128       udp dpt:5353 
ACCEPT     udp      ::/0                 ::/0               udp dpt:631 
ACCEPT     tcp      ::/0                 ::/0               tcp dpt:631 
ACCEPT     udp      ::/0                 ::/0               udp dpts:32768:61000 
ACCEPT     tcp      ::/0                 ::/0               tcp dpts:32768:61000     flags:!0x16/0x02 
ACCEPT     tcp      ::/0                 ::/0               tcp dpt:22 
ACCEPT     tcp      ::/0                 ::/0               tcp dpt:9002 
REJECT     all      ::/0                 ::/0               reject-with icmp6-adm-prohibited 
[root@SRV1 ~]# 

There is no config in both /etc/hosts.allow and /etc/hosts.deny files in SRV1.

[root@SRV1 ~]# cat /etc/hosts.allow 
#
# hosts.allow   This file describes the names of the hosts which are
#               allowed to use the local INET services, as decided
#               by the '/usr/sbin/tcpd' server.
#

[root@SRV1 ~]# cat /etc/hosts.deny 
#
# hosts.deny    This file describes the names of the hosts which are
#               *not* allowed to use the local INET services, as decided
#               by the '/usr/sbin/tcpd' server.
#
[root@SRV1 ~]# 

This is the tcpdump for non working scenario (i.e. SRV2 to SRV1).

[root@SRV1 ~]# tcpdump -i bond0.440 -nn -XX -vvv tcp port 9002 -s 0 -e
tcpdump: listening on bond0.440, link-type EN10MB (Ethernet), capture size 65535 bytes

21:55:49.016719 00:0c:29:95:fd:71 > a4:4c:11:de:38:b2, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl  64, id 4945, offset 0, flags [DF], proto: TCP (6), length: 60) 10.58.192.42.53766 > 10.58.192.36.9002: S, cksum 0x9a82 (correct), 2190603583:2190603583(0) win 5840 <mss 1460,sackOK,timestamp 2927224430 0,nop,wscale 7>
        0x0000:  a44c 11de 38b2 000c 2995 fd71 0800 4510  .L..8...)..q..E.
        0x0010:  003c 1351 4000 4006 9298 0a3a c02a 0a3a  .<.Q@.@....:.*.:
        0x0020:  c024 d206 232a 8291 f53f 0000 0000 a002  .$..#*...?......
        0x0030:  16d0 9a82 0000 0204 05b4 0402 080a ae79  ...............y
        0x0040:  e66e 0000 0000 0103 0307                 .n........
21:55:49.017588 a4:4c:11:de:38:b2 > 00:0c:29:95:fd:71, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 60) 10.58.192.36.9002 > 10.58.192.42.53766: S, cksum 0xb74c (correct), 1977460830:1977460830(0) ack 2190603584 win 5792 <mss 1460,sackOK,timestamp 3423009041 2927224430,nop,wscale 7>
        0x0000:  000c 2995 fd71 a44c 11de 38b2 0800 4500  ..)..q.L..8...E.
        0x0010:  003c 0000 4000 4006 a5f9 0a3a c024 0a3a  .<..@.@....:.$.:
        0x0020:  c02a 232a d206 75dd a85e 8291 f540 a012  .*#*..u..^...@..
        0x0030:  16a0 b74c 0000 0204 05b4 0402 080a cc06  ...L............
        0x0040:  f911 ae79 e66e 0103 0307                 ...y.n....
21:55:49.018224 00:0c:29:95:fd:71 > a4:4c:11:de:38:b2, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl  64, id 4946, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.42.53766 > 10.58.192.36.9002: ., cksum 0xfc88 (correct), 1:1(0) ack 1 win 46 <nop,nop,timestamp 2927224432 3423009041>
        0x0000:  a44c 11de 38b2 000c 2995 fd71 0800 4510  .L..8...)..q..E.
        0x0010:  0034 1352 4000 4006 929f 0a3a c02a 0a3a  .4.R@.@....:.*.:
        0x0020:  c024 d206 232a 8291 f540 75dd a85f 8010  .$..#*...@u.._..
        0x0030:  002e fc88 0000 0101 080a ae79 e670 cc06  ...........y.p..
        0x0040:  f911                                     ..
21:55:49.018923 a4:4c:11:de:38:b2 > 00:0c:29:95:fd:71, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl  64, id 20812, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.36.9002 > 10.58.192.42.53766: F, cksum 0xfc84 (correct), 1:1(0) ack 1 win 46 <nop,nop,timestamp 3423009044 2927224432>
        0x0000:  000c 2995 fd71 a44c 11de 38b2 0800 4500  ..)..q.L..8...E.
        0x0010:  0034 514c 4000 4006 54b5 0a3a c024 0a3a  .4QL@.@.T..:.$.:
        0x0020:  c02a 232a d206 75dd a85f 8291 f540 8011  .*#*..u.._...@..
        0x0030:  002e fc84 0000 0101 080a cc06 f914 ae79  ...............y
        0x0040:  e670                                     .p
21:55:49.019172 00:0c:29:95:fd:71 > a4:4c:11:de:38:b2, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl  64, id 4947, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.42.53766 > 10.58.192.36.9002: F, cksum 0xfc83 (correct), 1:1(0) ack 2 win 46 <nop,nop,timestamp 2927224432 3423009044>
        0x0000:  a44c 11de 38b2 000c 2995 fd71 0800 4510  .L..8...)..q..E.
        0x0010:  0034 1353 4000 4006 929e 0a3a c02a 0a3a  .4.S@.@....:.*.:
        0x0020:  c024 d206 232a 8291 f540 75dd a860 8011  .$..#*...@u..`..
        0x0030:  002e fc83 0000 0101 080a ae79 e670 cc06  ...........y.p..
        0x0040:  f914                                     ..
21:55:49.019185 a4:4c:11:de:38:b2 > 00:0c:29:95:fd:71, ethertype IPv4 (0x0800), length 66: (tos 0x0, ttl  64, id 20813, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.36.9002 > 10.58.192.42.53766: ., cksum 0xfc83 (correct), 2:2(0) ack 2 win 46 <nop,nop,timestamp 3423009044 2927224432>
        0x0000:  000c 2995 fd71 a44c 11de 38b2 0800 4500  ..)..q.L..8...E.
        0x0010:  0034 514d 4000 4006 54b4 0a3a c024 0a3a  .4QM@.@.T..:.$.:
        0x0020:  c02a 232a d206 75dd a860 8291 f541 8010  .*#*..u..`...A..
        0x0030:  002e fc83 0000 0101 080a cc06 f914 ae79  ...............y
        0x0040:  e670                                     .p

6 packets captured
6 packets received by filter
0 packets dropped by kernel
[root@SRV1 ~]# 

This is the tcpdump for working scenario (i.e. SRV1 to SRV1 itself).

[root@SRV1 ~]# tcpdump -i lo -nn -XX -vvv tcp port 9002 -s 0 -e
tcpdump: listening on lo, link-type EN10MB (Ethernet), capture size 65535 bytes

22:10:04.734084 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 74: (tos 0x10, ttl  64, id 31083, offset 0, flags [DF], proto: TCP (6), length: 60) 10.58.192.36.34764 > 10.58.192.36.9002: S, cksum 0xd5ff (correct), 624197334:624197334(0) win 32792 <mss 16396,sackOK,timestamp 3423864765 0,nop,wscale 7>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4510  ..............E.
        0x0010:  003c 796b 4000 4006 2c84 0a3a c024 0a3a  .<yk@.@.,..:.$.:
        0x0020:  c024 87cc 232a 2534 7ed6 0000 0000 a002  .$..#*%4~.......
        0x0030:  8018 d5ff 0000 0204 400c 0402 080a cc14  ........@.......
        0x0040:  07bd 0000 0000 0103 0307                 ..........
22:10:04.734177 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 74: (tos 0x0, ttl  64, id 0, offset 0, flags [DF], proto: TCP (6), length: 60) 10.58.192.36.9002 > 10.58.192.36.34764: S, cksum 0x0471 (correct), 3880130173:3880130173(0) ack 624197335 win 32768 <mss 16396,sackOK,timestamp 3423864765 3423864765,nop,wscale 7>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  003c 0000 4000 4006 a5ff 0a3a c024 0a3a  .<..@.@....:.$.:
        0x0020:  c024 232a 87cc e746 167d 2534 7ed7 a012  .$#*...F.}%4~...
        0x0030:  8000 0471 0000 0204 400c 0402 080a cc14  ...q....@.......
        0x0040:  07bd cc14 07bd 0103 0307                 ..........
22:10:04.734209 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl  64, id 31084, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.36.34764 > 10.58.192.36.9002: ., cksum 0xec94 (correct), 1:1(0) ack 1 win 257 <nop,nop,timestamp 3423864765 3423864765>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4510  ..............E.
        0x0010:  0034 796c 4000 4006 2c8b 0a3a c024 0a3a  .4yl@.@.,..:.$.:
        0x0020:  c024 87cc 232a 2534 7ed7 e746 167e 8010  .$..#*%4~..F.~..
        0x0030:  0101 ec94 0000 0101 080a cc14 07bd cc14  ................
        0x0040:  07bd                                     ..
22:10:04.737961 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 87: (tos 0x0, ttl  64, id 4547, offset 0, flags [DF], proto: TCP (6), length: 73) 10.58.192.36.9002 > 10.58.192.36.34764: P, cksum 0x9289 (correct), 1:22(21) ack 1 win 256 <nop,nop,timestamp 3423864769 3423864765>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0049 11c3 4000 4006 942f 0a3a c024 0a3a  .I..@.@../.:.$.:
        0x0020:  c024 232a 87cc e746 167e 2534 7ed7 8018  .$#*...F.~%4~...
        0x0030:  0100 9289 0000 0101 080a cc14 07c1 cc14  ................
        0x0040:  07bd 5368 6565 724d 6163 6869 6e65 496e  ..SheerMachineIn
        0x0050:  7465 7266 6163 65                        terface
22:10:04.737974 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl  64, id 31085, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.36.34764 > 10.58.192.36.9002: ., cksum 0xec77 (correct), 1:1(0) ack 22 win 257 <nop,nop,timestamp 3423864769 3423864769>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4510  ..............E.
        0x0010:  0034 796d 4000 4006 2c8a 0a3a c024 0a3a  .4ym@.@.,..:.$.:
        0x0020:  c024 87cc 232a 2534 7ed7 e746 1693 8010  .$..#*%4~..F....
        0x0030:  0101 ec77 0000 0101 080a cc14 07c1 cc14  ...w............
        0x0040:  07c1                                     ..
22:10:04.737986 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 68: (tos 0x0, ttl  64, id 4548, offset 0, flags [DF], proto: TCP (6), length: 54) 10.58.192.36.9002 > 10.58.192.36.34764: P, cksum 0xdf64 (correct), 22:24(2) ack 1 win 256 <nop,nop,timestamp 3423864769 3423864769>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4500  ..............E.
        0x0010:  0036 11c4 4000 4006 9441 0a3a c024 0a3a  .6..@.@..A.:.$.:
        0x0020:  c024 232a 87cc e746 1693 2534 7ed7 8018  .$#*...F..%4~...
        0x0030:  0100 df64 0000 0101 080a cc14 07c1 cc14  ...d............
        0x0040:  07c1 0d0a                                ....
22:10:04.737989 00:00:00:00:00:00 > 00:00:00:00:00:00, ethertype IPv4 (0x0800), length 66: (tos 0x10, ttl  64, id 31086, offset 0, flags [DF], proto: TCP (6), length: 52) 10.58.192.36.34764 > 10.58.192.36.9002: ., cksum 0xec75 (correct), 1:1(0) ack 24 win 257 <nop,nop,timestamp 3423864769 3423864769>
        0x0000:  0000 0000 0000 0000 0000 0000 0800 4510  ..............E.
        0x0010:  0034 796e 4000 4006 2c89 0a3a c024 0a3a  .4yn@.@.,..:.$.:
        0x0020:  c024 87cc 232a 2534 7ed7 e746 1695 8010  .$..#*%4~..F....
        0x0030:  0101 ec75 0000 0101 080a cc14 07c1 cc14  ...u............
        0x0040:  07c1                                     ..

7 packets captured
14 packets received by filter
0 packets dropped by kernel
[root@SRV1 ~]# 

Can you please suggest where the problem is?

nvh
  • 1
  • 1
  • 1
    Check your firewall settings allow port 9002. – user9517 Oct 05 '14 at 14:10
  • I have already tried disabling both iptables and ip6tables services on SRV1. But no luck... – nvh Oct 05 '14 at 14:16
  • Is your service `tcp_wrappers` aware? This looks quite a lot like what happens when `hosts.{allow,deny}` get involved. – MadHatter Oct 05 '14 at 15:20
  • There is no config in both /etc/hosts.allow and /etc/hosts.deny files in SRV1. – nvh Oct 05 '14 at 15:40
  • is your application accepting the connection (i.e. do you see the other machine connecting) before the connection gets closed? did you try to do a packet capture? – Ale Oct 05 '14 at 16:21
  • Yes, it accepts the connection and the TCP 3-way handshake also happens. But SRV1 sends a FIN/ACK immediately after the TCP 3-way handshake without sending any data. – nvh Oct 05 '14 at 16:31
  • pretty strange... but your application is not closing the connection as it actually happens? is it using any particular library for networking? – Ale Oct 05 '14 at 16:34
  • just to check, can you run a nc in listen mode on the same port instead of your application, and check if the connection also gets closes the same way? – Ale Oct 05 '14 at 16:36
  • I am not aware about the application that is running on SRV1 port 9002 since it is administered by someone else. So I have to check from them to stop it and run 'nc -l 9002' to check. But I tried an unused port on SRV1 by running 'nc -l 11112' without any modification to iptables or hosts.allow and it works from SRV2. – nvh Oct 05 '14 at 17:21
  • I managed to stop the service and ran Netcat on the same port. It worked like a charm. So the application admin agreed that there is no issue with RHEL but it may be an application issue. Thank you all of you for your advises/suggestions specially Ale. Appreciate your help. – nvh Oct 06 '14 at 15:10

0 Answers0