Remove Duplicate IP Addresses with Scapy

Question

I have a script to listen for incoming traffic and print out only the string “IP 1.1.1.1 53" when a packet hits the line. But now that I’m doing IP resolve on the IPs, I need to access the “ip_src” variable and only do the geolocation once on each ip, rather than resolve the same IP over and over as they come in. My current code is:

#!/usr/bin/python3
from scapy.all import *
import ipinfo

def print_summary(pkt):
  if IP in pkt:
     ip_src=pkt[IP].src
  if UDP in pkt:
     udp_sport=pkt[UDP].sport
     access_token = ''
     handler = ipinfo.getHandler(access_token)
     match = handler.getDetails(ip_src)
     c = match.details.get('city')
     s = match.details.get('region')
     strang = ("IP " + str(ip_src) + " " + str(udp_sport) + " " + str(c) + ", " + str(s))
     print(strang)

sniff(filter="",prn=print_summary)

As you can see the “print_summary” function is called by “prn” which is called for each pkt. I basically want to mimic the functionality of uniq and sort, since they can successfully filter out duplicates from a file, but I’d like to have it all in one script.

EDIT - Trying Set():

So using the code:

from scapy.all import *

def print_summary(pkt):
  if IP in pkt:
     ip_src=pkt[IP].src
  if UDP in pkt:
     udp_sport=pkt[UDP].sport
     lines_set = set(ip_src)
     strang = ("IP " + str(ip_src) + " " + str(udp_sport))
     if ip_src not in lines_set:
        for line in lines_set:
           print(line)
sniff(filter="",prn=print_summary)

I get the output: (in the terminal each character has a trailing newline)

2 . 3 5 8 0 1 2 . 4 8 0 1 . 6

This is not hard to do; what did you search for, and what did you find? What did you try, and how did it fail? Basically, use a `set` (or a `dict` if you like) to keep track of which addresses you have already seen. — tripleee, Oct 01 '20 at 05:07
Your changed code does something completely different; no wonder it doesn't work. You want to create a variable which persists between calls to the handler, and check if the IP address you are processing now is already in that set. — tripleee, Oct 01 '20 at 07:06
I rolled back your edit; your question should remain strictly a question. — tripleee, Oct 01 '20 at 07:51

tripleee · Accepted Answer · 2020-10-01T07:58:55.403

This adds a set variable to keep track of which addresses you have already seen.

#!/usr/bin/python3
from scapy.all import *
import ipinfo
 
seen = set()
 
def print_summary(pkt):
  if IP in pkt:
     ip_src=pkt[IP].src
     if UDP in pkt and ip_src not in seen:
         seen.add(ip_src)
         udp_sport=pkt[UDP].sport
         access_token = ''
         handler = ipinfo.getHandler(access_token)
         match = handler.getDetails(ip_src)
         c = match.details.get('city')
         s = match.details.get('region')
         strang = ("IP " + str(ip_src) + " " + str(udp_sport) + " " + str(c) + ", " + str(s))
         print(strang)
 
sniff(filter="ip",prn=print_summary)

I also changed the indentation of the second if to avoid getting a traceback if somehow you would receive a packet which doesn't have the IP member; though I also updated the filter expression to hopefully prevent that from ever happening in the first place.

Remove Duplicate IP Addresses with Scapy

1 Answers1