3

OK, I've scoured the 'net already for possible solutions and have found none. Maybe you guys can help me.

I have a Windows Server 2003 setup, with a fixed IP on a routed LAN, standard /24 network. The server works fine, with the exception of one rather important annoyance: It ARP scans the entire subnet (all 256 addresses except its own) every 20-30 seconds, in two bursts of 128 addresses 10-15 seconds apart. This causes unnecessary and excessive ARP traffic on my LAN (up to 40% of all packets if the LAN is not heavily used).

The requests sent are standard ARP discovery requests with a broadcast MAC and sequential IP addresses in the LAN subnet (not gratuitous). None of the other devices or machines on the subnet are displaying this behaviour (Win XP, Win7, several routers, etc.) so it is something specific to Win Server 2003.

How do I stop it from constantly scanning the subnet for new MAC/IP combinations? I've already tried manually setting the ARP Cache timings on the NIC interface (in the registry) to 600 seconds but the server completely ignores that, apparently.

Mark
  • 41
  • 1
  • 2
  • 5
  • What network card does the server have? I have seen crazy ARP stuff from Broadcom before (Sending arp bursts for IPs that are not even in the local network)... – Kyle Brandt Jan 13 '12 at 18:59
  • It's an integrated Dell NIC, identified as an Intel PRO 1000 MT – Mark Jan 13 '12 at 19:07

2 Answers2

4

That sounds utterly bizarre. Are you certain you don't have any "unwanted" software, or software that's performing an "unwanted" function on the machine?

Sequential scans sound like either a piece of malicious software doing network scans, or some type of misguided network "management" running scans. Either way, that's not a stock behavior of Windows Server 2003. I recall Kyle Brandt describing the problems he saw the Broadcom drivers causing spurious ARPs, but if I recall properly there wasn't even a hint of sequential scanning to those ARPs.

Microsoft Network Monitor 3 or Process Monitor may be able to pin down the process on the machine that's actually generating the traffic. I'd lean toward Network Monitor, first.

Evan Anderson
  • 141,881
  • 20
  • 196
  • 331
  • 1
    In addition to "unwanted" software there is a whole bunch of crappy "network monitoring" or "management" software which is doing quite the same in the course of host/device discovery. – the-wabbit Jan 13 '12 at 19:23
  • @syneticon-dj: True dat. I'll drop on an edit. – Evan Anderson Jan 13 '12 at 19:24
  • @syneticon-dj: Dell OpenManage used to do this 7-8 years ago. As I haven't used Dell servers since then I have no idea if this still is the case. SAN or SCSI or tape-library managment tools could also be involved. – Tonny Jan 13 '12 at 19:32
  • There is no unwanted software on the server afaik. Network Monitor gives little additional information. the ARP requests are lumped in under "" - However, it did show me something additional: the "known" addresses on the LAN aren't ARP probed, but are sent a UDP request with payload "get ip" to port 51220 instead in the scan. Really bizarre. – Mark Jan 13 '12 at 19:35
  • 1
    @Mark: NetMon 3.4 should show you the process ID of the process generating the traffic. (The stock version of NetMon in Server 2003 can't do that.) That's not stock behavior of the OS-- you've got something third-party doing that. – Evan Anderson Jan 13 '12 at 19:48
  • @Evan: Process ID (after adding the column) and Process Name are blank in NetMon 3.4 - I'm going through my third party programs now (a very short list) – Mark Jan 13 '12 at 19:52
  • OK, the issue has been found and squelched. The culprit is "ViewPower Pro" which is supplied with FSP UPSes, which I found by toggling services while keeping a close eye on networking. This turned out to be my UPS monitoring software's SNMP subroutine, Switching SNMP functionality off stopped the UDP and ARP flooding (UDP requests were also sent to unavailable IPs as it turned out, just less frequently). Since I'm not using SNMP for monitoring, it's not needed. Talk about a braindead piece of software ;) Evan, syneticon-dj, Tonny: thanks a lot for the pointers! – Mark Jan 13 '12 at 20:01
  • Glad I could help. That's quite an annoying behavior in that software! – Evan Anderson Jan 14 '12 at 05:19
  • Good thing you didn't use `10.0.0.0/8` for your LAN! – David Schwartz Jan 14 '12 at 05:49
0

Servers with Broadcom teaming and incoming load balancing enabled are known to generate arp storms. But the sequential scan really sounds suspicious.

Prevention of such ARP storm is possible but not practical since you will not be able to differentiate between a legitimate ARP traffic and a problematic one. You will have to identify which software is causing the problem, I cannot think of any scenario in which a server would need to relearn its entire ARP table every 30 seconds.

Benny
  • 181
  • 1
  • 7