1

We have a small network of around 100 laptop/desktops and around 20 servers (sounds an overkill but these servers provide service for all our external contractors too) and we have recently noticed that our Exchange CAS server has been hacked. The hackers installed VPN server and were using our network to go out again. We have also noticed that they installed number of Malwares. We have so far cleaned that machine however we have now got another problem with ARP attack.

There is a nonstop sequential arp request to all the ip's in the subnet and almost every minute our Juniper firewall (gateway) MAC is being swapped for our Juniper VPN device which as a result is making Internet access unavailable to staff members since the Barracuda box is the wrong box to be routing traffic.

Finally there is a nonstop request/reply of ARP to 0.0.0.0 and a none existing MAC address in my network.

We use Cisco 2800 router for the internet and all our internal switches are HP Procurve while we have a mix of Windows and Linux servers running in the network. Also, all our servers are Vmware Vsphere virtualised.

I'm so far reaching my limit in identifying the machine(s) causing this problem and any help ideas would be highly appreciated.

----Edit---

I have since run Wireshark on the network as you guys suggested and there is a weird pattern that I have noticed. My Juniper VPN device's MAC address is being allocated to my Juniper firewall which as a result is cutting off Internet connection and also Wireshark says my Barracuda box IS the source as can be seen from the this image below: https://i.stack.imgur.com/i7T0p.png

aniga
  • 21
  • 4
  • 5
    Run a packet capture. Look at the ARP request in the capture. Identify the source MAC address in the ARP request. Look for the switch port tied to that that MAC address in the MAC address table of the switch. Identify the device connected to that switch port. Inspect that device. – joeqwerty Sep 23 '14 at 15:48
  • @joeqwerty, you may want to move your comment to an answer, so it can be accepted. – zymhan Sep 23 '14 at 19:32
  • `Who has Tell 0.0.0.0` is a standard approach to avoiding IP address conflicts. The other option is to use `169.254/16` (i.e., RFC 3927), but most equipment I've seen uses the former. Thus, this particular part of the observed behavior is not, in itself, of concern. – BMDan Sep 24 '14 at 14:54

2 Answers2

4

Run a packet capture.

Look at the ARP request in the capture.

Identify the source MAC address in the ARP request.

Look for the switch port tied to that that MAC address in the MAC address table of the switch.

Identify the device connected to that switch port.

Inspect that device.

joeqwerty
  • 109,901
  • 6
  • 81
  • 172
  • Thanks for your time, I have now updated the question with what I have found since following your instructions in locating the source. I don't want to remove/disconnect our Barracuda spam filter as it is the first point of call for all our mail. Could that box be the culprit? – aniga Sep 24 '14 at 14:36
1

Given that there is potentially root-level malware on the loose, grab a known-good laptop (e.g. one booted from a LiveCD) and plug it into a port on the switch. Fire up some packet capturing software. Unplug cables (optionally in small groups) until the traffic of interest stops. Identify the device(s) connected to the cable(s) in question. Note that, in the event this is being triggered by a misconfiguration, there may be a pair of devices causing the issue, so if the configuration on the triggering device seems right, test it in isolation and/or in a reduced neighborhood in order to eliminate interplay as a concern.

Proper (i.e., non-malicious) ARP requests and replies contain MAC addresses, so you can also use those to affirmatively identify the involved machine(s).

One last note: depending on how smart your switch is, you may be able to force a particular MAC<->IP mapping onto it by adding a static ARP entry, whereupon you'll at least be able to track down this issue in some degree of peace.

BMDan
  • 7,249
  • 2
  • 23
  • 34
  • Thanks for your time, I have now updated the question with what I have found since following your instructions in locating the source. I don't want to remove/disconnect our Barracuda spam filter as it is the first point of call for all our mail. Could that box be the culprit? – aniga Sep 24 '14 at 14:37
  • The information you've provided would suggest that the Barracuda is configured with the same IP address as your firewall. Are you able to conclusively determine that that is not the case? Remember to check all zones/interfaces. – BMDan Sep 24 '14 at 14:52
  • Once again thanks for your time. Yes I can 100% guarantee that they are not configured with the same IP. Also, I don't know why I was so fixated in saying Barracuda Spam box when the box it is swapping IP with is Juniper VPN device. – aniga Sep 24 '14 at 15:25