1

I'm having a strange problem with an RHEL 5 server. I changed its IP address, we are migrating to a new address pool, and something is blocking all outgoing traffic, but I can SSH in on that interface (eth0). All traffic works properly on our private net (eth1). I flushed iptables so it should not be in the way. My routing table looks just like another RHEL 5 server that made the switch without problem. I have rebooted and no-joy there either. Where else might a latent old IP address be lurking that is causing this problem, because when I switch it back to the old address it works just fine.

EDIT: As requested here is the routing table and output of 'arp -n'

# route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
*****.56.0    *               255.255.255.0   U     0      0        0 eth0
10.0.0.0        *               255.255.255.0   U     0      0        0 eth1
default         gateway.***** 0.0.0.0         UG    0      0        0 eth0


# arp -n
Address                  HWtype  HWaddress           Flags Mask            Iface
******.56.254           ether   00:A0:D1:E4:58:0D   C                     eth0
10.0.0.203               ether   00:40:05:A0:21:93   C                     eth1
10.0.0.128               ether   00:21:5A:4C:1C:40   C                     eth1
10.0.0.129               ether   00:21:5A:47:36:80   C                     eth1
10.0.0.1                 ether   00:0A:CD:20:64:28   C                     eth1
10.0.0.130               ether   00:22:64:0D:A5:46   C                     eth1
10.0.0.125               ether   00:22:64:0D:B5:A0   C                     eth1
10.0.0.127               ether   00:22:64:0B:D1:B0   C                     eth1
10.0.0.126               ether   00:21:5A:4C:4B:14   C                     eth1
10.0.0.124               ether   00:22:64:06:F8:AE   C                     eth1

As I mentioned, the private network works fine and these machines can be seen in the arp table.

Erik Weiss
  • 11
  • 3
  • 1
    Is traffic to hosts on the local network blocked, or is it only traffic that passes through a router? – Flup Mar 08 '13 at 16:18
  • As an aside, flushing iptables doesn't clear policies; if you flush the OUTPUT chain but the chain policy is DROP, then there won't be **any** traffic output from the machine. I don't think that's what's biting you in this case, but bear in mind in future that flushing the chains isn't the way to take iptables out of the picture. – MadHatter Mar 08 '13 at 16:20
  • Something is filtering the network, either it's an unmatched netmask or something vlan based.. – NickW Mar 08 '13 at 18:25
  • I bypassed my VLAN and plugged into the wall instead of the switch to my ISP's network, same problem. I'm not sure how it could be netmask related if my netmask shows up correctly (255.255.255.0) in the routing table, and ifconfig. Besides /etc/sysconfig/networking/devices/... what other config file would have the netmask that I might be missing? – Erik Weiss Mar 08 '13 at 21:04

3 Answers3

0

Is your Gateway set properly for the new IP? Are you on the same subnet as the previous IP? What do you mean by outgoing traffic? Do you mean anything past the switch, or do you mean you cannot ping a computer on the same subnet? If it is the first, check your Gateway settings.

  • Ah yes sorry for not being specific. The new IP is on a different subnet, and different wiring all together. As well, I cannot ping a local ip address on the same subnet. – Erik Weiss Mar 08 '13 at 16:35
  • @ErikWeiss do you have any VLANs set up, especially on the switch side? – NickW Mar 08 '13 at 16:40
  • Yes, I do. Its actually the switch this server is plugged into. Saves me the time of walking downstairs to switch from one LAN to another. Could this be causing the problem? Other servers don't complain about it. – Erik Weiss Mar 08 '13 at 16:46
  • @ErikWeiss Is it untagged? Tagged? Is the port you're plugging into the same port as the old subnet, or is it a new port? – NickW Mar 08 '13 at 17:05
  • Untagged. I leave it in the same port and then switch which VLAN the port is assigned to. Then on the server I do, 'ifdown eth0' 'ifup eth0'. I've tried this routine with a few different ports on the switch to eliminate bad ports as the problem. – Erik Weiss Mar 08 '13 at 17:13
  • So the server is unaware of the VLAN, ok. What about arp tables, are arps resolving at all? – NickW Mar 08 '13 at 17:22
  • Here is a the output of 'arp -n' # arp -n Address HWtype HWaddress Flags Mask Iface ******.56.254 ether 00:A0:D1:E4:58:0D C eth0 I *'d out the real IP address since it is a public IP. But it is the IP of the internet gateway. I verified the MAC address. – Erik Weiss Mar 08 '13 at 17:31
  • Maybe you can drop that + the routing table (suitably **'d where necessary) in the original question? – NickW Mar 08 '13 at 17:45
0

Can you reach the gateway from the box in question? I assume that you can ping 127.0.0.1 and the configured ip address. If you have access to the switch, you should be able to the mac of the nic and confirm the vlan in question (assuming you can reach the switch from the box).

Can you confirm that the netmask is correct? I would also draw your attention to any configurations regarding selinux and iptables, (but it sounds like you have been poking there already).

ewm
  • 139
  • 6
  • Yup, selinux is disabled, and iptables is flushed and the default policy is accept. (Not what I usually keep it on). I cannot ping the gateway from the machine in question. I poked around on the switch and I don't see where I can tell the MAC of a nic plugged into a port from the switch. I'm have a Procurve 2848. Oh yes and the netmask is 255.255.255.0, which is correct and works on all the other machines on the new subnet. – Erik Weiss Mar 08 '13 at 17:22
  • run 'ethtool eth0' - look at the last line; it will say 'Link detected:' with a yes or no. If yes, you have a link and configuration is the issue, if no, there is media problem. I'm not familiar with the Procurve but the cisco command is 'show mac address-table' and google points to the 'show mac-address' command for the Procurve. – ewm Mar 08 '13 at 17:54
  • Link Detected is yes, on both VLANs. (I switched it to see if it changed when I changed the VLAN) show mac-address shows the MAC of eth0 – Erik Weiss Mar 08 '13 at 18:25
  • If you were using ifconfig to bring the interfaces up and down after the changes, and it is still not working, you may want to try 'service network restart' - some changes aren't pickup via ifconfig up. – ewm Mar 12 '13 at 12:57
0

It took an upgrade (not recommended, do a fresh install) to figure out it was iptables after all. Doing iptables -F was not enough, it took service iptables stop to actually clear out that stale rule.

Erik Weiss
  • 11
  • 3