Questions tagged [netflow]

is a Cisco standard IP traffic reporting format, summarizing network sessions in terms of endpoints, port/protocol, bytes transferred, and other data.

NetFlow is a standard network session reporting format, developed by Cisco to summarize network communications between hosts. This data generally includes:

  • endpoint IP addresses, including the initiator
  • IP protocol (by IANA number)
  • port where applicable (generally only TCP or UDP)
  • bytes transferred according to direction

and other data. (A fuller listing can be found at the Wikipedia article on NetFlow)

A NetFlow exporter (which may include a router, switch, or other device; or a software exporter sniffing local traffic) maintains a list of tracked connections by analyzing the headers of packets visible to it. NetFlow does not require deep packet inspection; by default it summarizes only IP header and protocol header information. The exporting device periodically exports information on these connections to "collector" software, usually via UDP at the default port 2055. Such a collector may receive updates from multiple exporters on a push basis, with each datagram reporting identifying information for its exporter. Each datagram can include multiple sets of NetFlow sessions.

A number of free/OSS and commercial products are available to collect and analyze NetFlow data, as well as similar data sources such as sFlow and JFlow. Such products may support storage and search, monitoring for security purposes, traffic visualizations, network-based forensic procedures, or statistical analysis. For development, flowd is an open source flow collector implementation.

There are multiple versions of NetFlow, but NetFlow 5 and NetFlow 9 are the predominant ones. NetFlow 5 is a fixed format, with the same fields in the same position for every export. NetFlow 9 has a range of optional fields, some of which are vendor-specific. In order to decode a NetFlow 9 datagram, it is necessary to match the datagram to a numbered template (templates are exported in the datagram stream periodically). Each template will contain an in-order listing of the items to be found in NetFlow records exported according to that template. The NetFlow 9 standard is described in RFC 3954.

NetFlow 9 is the basis of the IPFIX standard, and has been adapted as the basis for formats such as NSEL (NetFlow Security Event Logging). Because IPFIX is built on NetFlow 9, libipfix may be useful to NetFlow developers.

101 questions
1
vote
2 answers

Is there an XML schema for description net flow data?

I am working on a program that imports and exports net flow data. Because of our general architecture, it would make sense to describe this information in XML. Does there exist an XML for describing net flow data? A good description of NetFlow data…
vy32
  • 28,461
  • 37
  • 122
  • 246
1
vote
0 answers

Convert Elastiflow network flow into 3D graphics

I am looking for a way to show Elastiflow's Netflow traffic in 3D graphics and I would like to know a solution that I could implement. Flow Trafic Like: The 3D presentation I would like to achieve would be something like this or similar.
withesoc
  • 11
  • 4
1
vote
1 answer

How to replace values to binary(0-1) in Pandas for Network data?

I have 75 columns, and 300k captured network traffic CSV file. I am playing with data to apply ML. I need to convert IP addresses to 1 and 0 according to internal and external. So if it is 10.0.2.* > 0 others > 1 Is there an easy way to do this? I…
1
vote
2 answers

Creating new columns with Pandas df.apply

I have a huge NetFlow database, (it contains a Timestamp, Source IP, Dest IP, Protocol, Source and Dest Port Num., Packets Exchanged, Bytes and more). I want to create custom attributes based on the current and previous rows. I want to calculate new…
chad
  • 33
  • 4
1
vote
1 answer

Is this how to use Scapy to parse a Netflow packet?

I'm new to scapy and am looking for confirmation that the commands below is a valid approach to dig into (parse) a UDP/NetflowV9 packet. My reason for asking is that the approach seems valid to me and works for many packets that I've tried. But, it…
wwwalker
  • 31
  • 8
1
vote
0 answers

How to calculate the min. and max packet size of a flow

I am trying trying to get the minimum, maximum packet size of a flow and also the inter-arrival time between the packets from a network traffic which I captured. I have tried but i did not make any progress. The code below: 1. Counts the number…
TeeGee
  • 19
  • 1
  • 6
1
vote
0 answers

How to read dump flow-capture from GO?

The capture stream adds netflow v5 dumps to the server, when reading it from GO it is either simply impossible to read, because the first two bytes are not a version of netflow, or if I transfer the packet to 11 arrays, it gives the packet…
1
vote
0 answers

Optimization of a Packet Generation Script

I have a dataset full of 250+ million entries of netflow data. My goal is to develop an efficient way of generating packets for this netflow data, and I've decided to use Scapy as my means of emulating packets. For each of the entries in my dataset,…
NolanRudolph
  • 106
  • 11
1
vote
1 answer

Netflow V9 field Id range

I am confused about the range of field IDs that are supported in netflow v9. I am getting varied data from online sources from 79, 127, 128. I got the above information from (79) - NetFlow v9 has a set of 79 field types defined, whereas IPFIX has…
Kingkong Jnr
  • 1,128
  • 4
  • 18
  • 29
1
vote
1 answer

Decoding IPFIX packets using Python

I want to parse the ipfix packet over udp from openvswitch, Is there a lib in python 2.7 could help me do this? Thanks
Frank Wang
  • 41
  • 1
  • 5
1
vote
1 answer

Logstash Netflow Module listening, but not reading packets

Logstash 6.2.4 with the netflow module Elasticsearch Version: 6.2.4 Ubuntu 16.04 LTS I have an issue where logstash is listening on the correct port, but does not seem to be collecting the netflow data and passing it to elasticsearch. The routers…
1
vote
1 answer

Nfdump nfcapd file - binary to csv file conversion

Given a nfcapd file produced in nfdump nfcapd.2017 which is in the default binary format How would I create a version of this file in csv format using nfdump? I tried using nfdump -r nfcapd.2017 -w newfile -o csv but that doesn't seem to work
crazyCoder
  • 1,552
  • 3
  • 20
  • 25
1
vote
0 answers

How to parse/decode raw Netflow data to text in Java?

I want to convert raw Netflow traffic data to human readable format in java. Does any one have any clue how to achieve this.
1
vote
1 answer

NetFlow timestamps

What's the best way to determine the absolute time for the flows in a NetFlow data packet? It looks like only relative time information is included in the flows (SysUpTime on start and end of the flow). For Netflow v5 and v9, it's possible to…
TWT
  • 2,511
  • 1
  • 23
  • 37
1
vote
1 answer

NetFlow, count min, median, and max number of bytes per flow

I have captured .silk file. Is it possible to count min, median, and max number of bytes per flow using SiLK utils? If yes, could you please point me in the right direction.
iceone213
  • 1,141
  • 1
  • 11
  • 22