1

I'm new to scapy and am looking for confirmation that the commands below is a valid approach to dig into (parse) a UDP/NetflowV9 packet.

My reason for asking is that the approach seems valid to me and works for many packets that I've tried. But, it fails for at least one packet that I've encountered. The decode/parsing that fails is for a UDP/Netflow packet that has 4 Flowsets as follows:

  • Flowset 1: Data flowset that has 2 flow records
  • Flowset 2: Options Template flowset
  • Flowset 3: Data flowset that has 1 flow record
  • Flowset 4: Data template flowset

Approach

pkt = rdpcap(pcap)
pkt[0][UDP].payload  
NetflowHeader(pkt[0][UDP].payload)  
NetflowHeader(pkt[0][UDP].payload).payload  
NetflowHeaderV9(NetflowHeader(pkt[0][UDP].payload).payload)
NetflowHeaderV9(NetflowHeader(pkt[0][UDP].payload).payload).payload
NetflowDataflowsetV9(NetflowHeaderV9(NetflowHeader(pkt[0][UDP].payload).payload).payload)
NetflowDataflowsetV9(NetflowHeaderV9(NetflowHeader(pkt[0][UDP].payload).payload).payload).payload
NetflowOptionsFlowsetV9(NetflowDataflowsetV9(NetflowHeaderV9(NetflowHeader(pkt[0][UDP].payload).payload).payload).payload)

The commands above seem to work to reveal details about the first 2 flowsets, but the output of the last command has wrong data (doesn't match what Wireshark shows) for flowset 3. In particular, to start off with, for flowset 3 the flowsetID and length are wrong. I haven't even yet tried to look at flowset 4 since the data for flowset 3 is wrong.

So, I'm just looking for confirmation that my appoach is correct. If my approach is correct, then I'll try to find a way to anonymize the pcap file (a single frame) and ask in a separate stackoverflow post(s). I'd like to not reveal IP addresses in the flow records.

wwwalker
  • 31
  • 8
  • Have you read the doc? https://scapy.readthedocs.io/en/latest/layers/netflow.html – Cukic0d Jun 01 '20 at 00:11
  • Yes, but possibly I didn't grasp something. The section on parsing/dissecting shows just two bullet points. I tried using a command/method from one of the bullet points "netflowv9_defragment" and it did decode the Eth/IP/UDP from the frame but all the Netflow content was shown just as hex values like this: ` – wwwalker Jun 01 '20 at 01:27
  • You'll need to give us some code to work with :/ What are the exact calls you're doing ? What did you try... Did you bind the Netflow headers to the UDP port you are using or was it the default one ? (it doesn't seem so in your first example) – Cukic0d Jun 01 '20 at 06:24
  • Sure, I'll work on that. I need to anonymize the pcap, check that the behavior I'm seeing remains, and then upload it somewhere. – wwwalker Jun 01 '20 at 15:03
  • 1
    @Cukic0d, thanks very much for the help. Upon anonymizing my netflow pcap, I decided to also change the dport to 2055. After that, I was able to use netflowv9_defragment(pkt)[0] to parse the packet. Similarly and in addition, after doing the bind_layers( UDP, NetflowHeader, dport=9990 ), netflow_v9_defragement also parsed the original pcap too. – wwwalker Jun 02 '20 at 20:15

1 Answers1

0

The answer to my original question was that I was not properly using scapy to parse Netflow traffic from a pcap.

I believe that the proper way to do it (thanks to help from @Cukic0d) is to:

  1. Bind the dest port of Netflow traffic in your pcap to the default Netflow port 2055 via bind_layers( UDP, NetflowHeader, dport=xxxx ) where xxxx is the dest port of Netflow traffic in your pcap. This is unnecessary if your Netflow traffic is already on 2055. Possibly it's also unnecessary for 2056.
  2. Read in the pcap (all at once or frame by frame). To read it in frame by frame, do this in a loop: pkt = rdpcap(pcap) where pcap is the path to your pcap file.
  3. Use netflowv9_defragment to parse the complete single netflow frame just read in like this: netflowv9_defragment(pkt)[0]
wwwalker
  • 31
  • 8
  • Note you can also use (my favorite), `sniff(session= NetflowV9Session)`. For instance, to read a pcap: `sniff(session=NetflowV9Session, offline="q.pcap")`. Also you are supposed to call the defragment function on the whole list of packets (the whole pcap), as some layers can be sent in other packets. – Cukic0d Jun 04 '20 at 00:30
  • ok, thanks for the more info. The reason that I had used rdpcap is that my pcap is huge, 500,000 frames. I figured that I should not try to deal with the whole file at once time due to memory limitations. Any guidance on that aspect? – wwwalker Jun 05 '20 at 01:10
  • `sniff` is more efficient than `rdpcap` to do that. You can also use `sniff` with `prn=...` and `store=0` – Cukic0d Jun 05 '20 at 06:16
  • 1
    I'm puzzled what is NetflowV9Session in the sniff command. When I try it, I get: NameError: name 'NetflowV9Session' is not defined – wwwalker Jun 05 '20 at 19:24
  • how can i print src,dest and bytes of IPFIX flow with scapy and python ? – Morteza Soltanabadiyan Oct 20 '20 at 14:11
  • @wwwalker try `sniff(session=NetflowSession, offline="mypcap.pcap")` – Jonathan Sep 29 '22 at 22:08