0

Current suspect is the Watchguard Firewall. Most likely I am to blame as the person who edited the firewall rules.

  1. At a recent point in the past my company was on a different external IP address. I switched providers and thus changed external IPs. Prior to the change I could ping from the mailserver and get the responses without fail.

  2. When I changed the firewall the only changes I made were to rules that specifically mentioned an external address. The physical connections for the internal network didn't change. The structure of the firewall rules didn't change (or so I thought).

  3. After changing the firewall and plugging into the new T1s I don't receive ping replies from 192.168.0.4 which is the mailserver's internal address. I noticed that outgoing mail was queuing up. Eventually I setup a second network interface on 192.168.0.25 and as soon as I enabled that interface all outgoing mail left the queues and ping responses came back to the mailserver.

  4. The new external IP addresses were listed on one of SORBS lists. I got an exclusion from the blacklist through an automated form for the individual external address of the mailserver. I found that getting the exclusion worked for some configurations but some recipient servers would still track the packet back to the firewall address which is still on the blacklist.

  5. I've found now that if I change the firewall rule to allow SMTP out from 192.168.0.4 instead of using the "filtered SMTP" rule that worked previously my outgoing mail is seen on the excluded external address and the recipient servers that use SORBS don't block the mail.

  6. Given that mail from 192.168.x.4 is what I'm allowing out on the external IP that isn't blacklisted I disabled the interface related to 192.168.0.25.

  7. Mail works now but ping replies are lost. Watching the firewall logs shows the outgoing ping as allowed and any IP in the 192.168.0.* range other than 192.168.0.4 receives the ping responses.

  8. Enabling the 192.168.0.25 interface ends up with pings working but mail being blacklisted by SORBS users. I can just enable it temporarily to ping and then disable it again but I'd rather solve the issue than just work around it all the time.

Edit: To be clear the only issue I need help with is why Ping replies don't make it back to 192.168.0.4 or why the server is dropping those responses. I do not need help with the SORBS or general mailserver issues. All issues relating to mail were resolved in the past I'm just looking for a solution to being able to ping from that server to an external address as needed.

Example behaviour:

C:\>ping www.google.com

Pinging www.l.google.com [74.125.65.147] with 32 bytes of data:

Request timed out.
Request timed out.
Request timed out.
Request timed out.

Expected behaviour:

C:\>ping www.google.com

Pinging www.l.google.com [74.125.65.99] with 32 bytes of data:

Reply from 74.125.65.99: bytes=32 time=22ms TTL=52
Reply from 74.125.65.99: bytes=32 time=22ms TTL=52
Reply from 74.125.65.99: bytes=32 time=22ms TTL=52
Reply from 74.125.65.99: bytes=32 time=22ms TTL=52

Ping statistics for 74.125.65.99:
    Packets: Sent = 4, Received = 4, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
    Minimum = 22ms, Maximum = 22ms, Average = 22ms

I haven't gotten a chance to sniff the packets but I did notice this

Routes:
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
192.168.0.199   -               255.255.255.255 !H    0      -        0 -
192.168.0.200   -               255.255.255.255 !H    0      -      668 -

The PC at IP 0.200 couldn't browse the web, get windows updates, or ping. I changed it's IP and now it can. I'm not sure how to clear these entries or why they were added in the first place. I don't see a similar route statement blocking traffic to/from 0.4

OK, I used NTOP to monitor the pings going to and from and I get this

296 bytes sent 0 bytes received 4 echo requests sent / 0 echo replies received.

I guess that means it's a firewall issue.

pplrppl
  • 1,262
  • 2
  • 14
  • 23
  • You have two separate issues here. One is that with one configuration, you're able to reach your mailserver, and with a different one that should work, you can't. The other problem is your SORBS issue. You'll have to deal with that separately. In terms of solving the first problem, you'll need to provide more information about your network topology and how everything is hooked up before we can help. – Schof Nov 10 '09 at 01:02
  • No, The mailserver works no matter what now. Sorbs isn't the issue and outgoing mail works no matter what now. The only remaining issue is why pings don't work. – pplrppl Nov 10 '09 at 01:29
  • It's worth noting that the addresses it resolves to seem to be valid addresses and can be pinged from another PC or from the same server if done from any IP other than 192.168.0.4 – pplrppl Nov 10 '09 at 01:45
  • OK, I guess joeqwerty gets the kudos for making me packet sniff. I haven't solved the issue but I have eliminated the Windows side of the house from the suspect list. – pplrppl Nov 12 '09 at 21:10

2 Answers2

1

Did you check the NAT configuration on the firewall to account for the new ip addresses? It sounds like a NAT or an ARP issue to me. Look at the NAT rules and the ARP table on the firewall and see if there's an entry that corresponds to the MAC address of the NIC in the server with the .4 ip address.

joeqwerty
  • 109,901
  • 6
  • 81
  • 172
  • I don't think this is an ARP issue. It would be much more obvious - no local connections inbound or outband to this machine would work (including SSH, local ping). – Yves Junqueira Nov 10 '09 at 01:48
  • The mailserver has a 1:1 nat in the firewall. I did just notice that the secondary DNS on the firewall didn't get repointed from .4 to .3 when I moved DNS off the mailserver. I just changed that and it didn't help. – pplrppl Nov 10 '09 at 02:00
  • Well I've seen some pretty strange ARP problems on Watchguard firewalls. Have you looked at the ARP entires on the firewall? Do the ARP entries for the server look correct? Are the ARP entries for the server on the correct firewall interface? Have you tried flushing the ARP cache on the firewall? I'm assuming the firewall is configured in routed (static) mode and not drop-in mode, is that correct? – joeqwerty Nov 10 '09 at 02:19
  • Ok, thanks for the ARP table info. Does the server's ip address show up in the ARP table for any other interface? Also, you're pretty adamant that the server is dropping the ping replies but I'm not convinced about it. Can you put a network sniffer on the server or between the server and firewall and verifiy that the ping replies are being sent to and getting to the server? They have to get to the server in order for the server to drop them and I'm not convinced that they're actually getting there. – joeqwerty Nov 10 '09 at 02:52
  • Also, are you using Automatic Proxy ARP on the firewall? I've seen this cause problems on Watchguard firewalls too. – joeqwerty Nov 10 '09 at 02:59
  • the arp table was very short, no goofiness there that I saw. From what I can tell the arp table is flushed when the firewall is rebooted and it gets rebooted when I update the rules most times. As to automatic proxy ARP I'll have to check that when I get back to work tomorrow. I don't know if the replies are getting back to the server, I just know the log on the firewall doesn't say it is blocking them (and it does show blocking other ping requests from unrelated hosts). Sniffing is the only way to be sure I suppose so that is a task for another day. – pplrppl Nov 10 '09 at 03:31
  • Apparently Automatic Proxy Arp is only available in drop in mode so No I'm not using it. – pplrppl Nov 10 '09 at 22:21
0

74.125.65.99 is a Google IP according to whois. So this is not a DNS problem.

Are you sure you have proper NAT rules configured on your gateway/firewall? It may not be translating the ping requests from the mailserver.

Yves Junqueira
  • 691
  • 4
  • 8