1

Problem is bizarre, at least to me.

I have a windows 2003 dedicated server. Every now and then (approx every three months), Cisco switch disconnects this server like so:

%PORT_SECURITY-2-PSECURE_VIOLATION: Security violation occurred, caused by MAC address 2020.2020.3c64 on port FastEthernet0/33.

ISP's support is trying to convince me that I have some malware that is trying mac spoofing attack, and that it is their policy to allow 3 mac addresses per port and that this occurs when this address limit is exceeded.

I've scanned with three different tools (including Microsoft's) and I can not find anything. I've checked web access logs around time when this happens, and there's not even script kiddies looking for phpmyadmin.

Could it be that some windows component is doing this? Any, and I do mean ANY advice on what to check next would be appreciated.

Chealion
  • 5,733
  • 28
  • 29
vnuk
  • 506
  • 1
  • 8
  • 17
  • Is the server the only thing connected to port Fa0/33? – Joseph Jun 24 '09 at 15:36
  • I don't know and have no choice but to assume yes. – vnuk Jun 25 '09 at 09:53
  • 1
    Have you checked to see if that is your servers MAC address? Also check within vmware to see if that MAC is anywhere at all. If you have 1 virtual machine you're already at 2 potential MAC addresses with the hardware MAC and the virtual MAC. If this has caused downtime I'd ask the ISP to move you to 4 MACs while you troubleshoot the cause. – sclarson Jun 30 '09 at 19:12

7 Answers7

1

The message indicates that the switch is seeing too many different ethernet source addresses on the port. There are a number of possible legitimate reasons for the machine to try to use more than one ethernet address per physical interface.

Are you running any virtual machines on your server (VMware or similar)? That may cause legitimate multiple MACs.

Are you running any high-availability software? Some methods used for resiliency relies on having one MAC for the physical machine and one MAC used for a service IP that floats between machines (VRRP and HSRP are two protocols to facilitate this).

Have you made sure that you don't have any hardware or driver issues? As radius says, malfunctioning drivers or hardware can cause this sort of behaviour.

Have you scanned the machine from a "known-clean boot"? It may be that you have a malware infestation that's clever enough to hide from your scan tools, but inspecting the disk from a known-good host (or a clean boot, off read-only media) MAY circumvent the method(s) used to hide.

Vatine
  • 5,440
  • 25
  • 24
  • vmware server is installed, but not used at all (even vmware's network interfaces are disabled). Machine is loaded with HP's software (since it is a rented dedicated server), but I somehow doubt that any of it would qualify as HA. No problems have been observed that would indicate hardware faults. I haven't scanned from clean boot because I can't afford that kind of downtime (serving three web sites). But thats an idea worth checking out... – vnuk Jun 17 '09 at 19:50
  • just scanned the machine from safe mode - no malware found – vnuk Jun 22 '09 at 21:20
0

What kind of network card you use and what king of driver is used ? I already see this kind of problems (but not on windows) where per a driver problem a buffer on the ethernet card was full and the card was sending packet with wrong information (including source mac address)

radius
  • 9,633
  • 25
  • 45
  • Server is HP DL320g5p, network card is onboard HP's NC324i. Driver is dated 22.7.2007, version 10.42.0.0 – vnuk Jun 17 '09 at 19:38
  • Latest version is 12.0.0.5. You may considere to update it http://h20000.www2.hp.com/bizsupport/TechSupport/SoftwareIndex.jsp?lang=en&cc=ch&prodNameId=3660063&prodTypeId=329290&prodSeriesId=3660062&swLang=8&taskId=135&swEnvOID=1005 – radius Jun 17 '09 at 22:09
  • Fascinating. I was looking at that page yesterday and didn't see it. Thanks, I'll install it tonight – vnuk Jun 18 '09 at 05:19
0

Try using Microsoft Network Monitor (it's free) and applying rule - "not UDP.Adress = ". You will see who's sending malformed frames.

jpou
  • 11
  • 1
0

Try an ipconfig /all how many network interfaces do you see?

Are you running any type of load balancing/clustering applications?

Is there anything special about your NIC? Multi-port, etc?

Have you tried a driver update for your NIC?

Can your hosting company give you the port security log? Looking at the MAC addresses that it's trying to use may give you a hint. You can look up the vendor id here. (considering that 20-20-20 is not registered to anyone, something is going on ...)

Joseph Kern
  • 9,899
  • 4
  • 32
  • 56
  • Ipconfig shows two interfaces (the other is an additional network card used for private connection to db server). As I can see, no driver update is available. Nic has two ports, one of them is used for iLo connection (different switch, different subnet). Will check for that port security log. – vnuk Jun 17 '09 at 19:44
0

If possible, run a protocol analyzer on that server such as Wireshark

That should at least give you some insight as to where the extraneous MAC addresses are coming from.

If it really is coming from your server, then you may have to look deeper. That MAC address doesn't seem valid at all, and you may have a process sending out frames that shouldn't be.

I would also verify that your physical connection to the provider switch doesn't go through any intermediary switches which may be injecting frames into the wrong ports.

Alo
  • 240
  • 1
  • 7
0

The ISP should be able to tell what MAC addresses it sees when this happens. They should also be able to configure the port to only allow the MAC address you want to work and discard any other MAC addresses instead of shutting down the port.

Joseph
  • 3,797
  • 27
  • 33
  • Your idea is great, and I suggested it to them. I got this: yes we could do that but we started this (blocking) practice when first wave of Conficker hit infected machines were using spoofed gateway mac address to sniff outgoing traffic. Does this make any sense or can I respond to it with something meaningful? – vnuk Jun 24 '09 at 15:02
0

0x202020203c64 is " >d", kind of a suspicious MAC address. Broken network card corrupting buffers, broken drivers or broken memory.

Run memtest86 on the server.

Or as they said, malware.

Thomas
  • 1,476
  • 11
  • 16