Questions tagged [netflow]

is a Cisco standard IP traffic reporting format, summarizing network sessions in terms of endpoints, port/protocol, bytes transferred, and other data.

NetFlow is a standard network session reporting format, developed by Cisco to summarize network communications between hosts. This data generally includes:

  • endpoint IP addresses, including the initiator
  • IP protocol (by IANA number)
  • port where applicable (generally only TCP or UDP)
  • bytes transferred according to direction

and other data. (A fuller listing can be found at the Wikipedia article on NetFlow)

A NetFlow exporter (which may include a router, switch, or other device; or a software exporter sniffing local traffic) maintains a list of tracked connections by analyzing the headers of packets visible to it. NetFlow does not require deep packet inspection; by default it summarizes only IP header and protocol header information. The exporting device periodically exports information on these connections to "collector" software, usually via UDP at the default port 2055. Such a collector may receive updates from multiple exporters on a push basis, with each datagram reporting identifying information for its exporter. Each datagram can include multiple sets of NetFlow sessions.

A number of free/OSS and commercial products are available to collect and analyze NetFlow data, as well as similar data sources such as sFlow and JFlow. Such products may support storage and search, monitoring for security purposes, traffic visualizations, network-based forensic procedures, or statistical analysis. For development, flowd is an open source flow collector implementation.

There are multiple versions of NetFlow, but NetFlow 5 and NetFlow 9 are the predominant ones. NetFlow 5 is a fixed format, with the same fields in the same position for every export. NetFlow 9 has a range of optional fields, some of which are vendor-specific. In order to decode a NetFlow 9 datagram, it is necessary to match the datagram to a numbered template (templates are exported in the datagram stream periodically). Each template will contain an in-order listing of the items to be found in NetFlow records exported according to that template. The NetFlow 9 standard is described in RFC 3954.

NetFlow 9 is the basis of the IPFIX standard, and has been adapted as the basis for formats such as NSEL (NetFlow Security Event Logging). Because IPFIX is built on NetFlow 9, libipfix may be useful to NetFlow developers.

101 questions
0
votes
0 answers

Troubleshooting Netflow Visibility Issue in Kibana

good time to you. I have a Cisco 2960 switch and I want to receive netflow data from this switch in Kibana so that I can design a dashboard for the received netflow. I was able to do this easily in GrayLog, but in Kibana, although netflow is being…
0
votes
0 answers

How to build multi v1/v5 records in one NetFlow packet#

From current scapy document and code,it seams there is no appropriate API to do this. Ihope to add the API to build multi v1/v5 records in one NetFlow packet. We need to build multi NetFlow records in one packet to simulate and test the packets just…
0
votes
0 answers

Unable to get snmp oid values and Netflow data to logstash 7.17

I have a Rocky Linux 8.7 VM with Elasticsearch, Kibana and Logstash (ELK stack using version 7.17). What I am trying to do is to have Logstash receive SNMP oid values and Netflow data from my mikrotik router, transfer them to Elasticsearch and from…
0
votes
0 answers

NetFlow Collector Agent

I have a problem with my python code that is about a Netflow collector agent. I run this code in virtual machine and must collects data from netflow agent that is a router which emulate in Gns3. My topology is correct and my devices can ping each…
0
votes
1 answer

How do I update specific Netflow v10/ IPFIX flow data fields in Python3 Scapy?

Scenario I have a PCAP containing Netflow v10/ IPFIX data templates and data flows and would like to replay the PCAP. Before the packet is sent, I would like to update one of the flow data fields (i.e. startTime with the current time). Current…
Jonathan
  • 148
  • 5
  • 21
0
votes
2 answers

How to count the same rows between multiple CSV files in Pandas?

I merged 3 different CSV(D1,D2,D3) Netflow datasets and created one big dataset(df), and applied KMeans clustering to this dataset. To merge them I did not use pd.concat because of memory error and solved with Linux terminal. df =…
0
votes
1 answer

Netflow application ID to application name

Im getting netflow data from multiples machines using filebeat, one of the fields I get is netflow.application_id and I need to transate the Id to the name of the application, the format of the application_id is this: 3, 0, 1, 244 four comma…
0
votes
1 answer

Understanding filebeat monitoring stats when ingesting netflow traffic

I'm running filebeat 7.14.0 to ingest Netflow data, which is then stored in Elasticsearch and viewed on Kibana. When I run filebeat -e, I will see some logs generated by filebeat every 30s. I'm trying to understand the stats more. For example, I…
Rayne
  • 14,247
  • 16
  • 42
  • 59
0
votes
1 answer

Logstash - Logstash stopped processing because of an error: (SystemExit) exit to install netflow

I am going to install NetFlow. Here is a document for it. My logstash.yml setting is following. modules: - name: netflow var.input.udp.port: 9996 I've run this command. /usr/share/logstash/bin/logstash --modules netflow -M…
grudev
  • 485
  • 1
  • 6
  • 15
0
votes
1 answer

Impossible to stop Logstash

I am using ELK stack with Netflow module. First of all, when I checked CPU usage Logstash was using a lot of resources and I decided to stop it. This moment Elasticsearch/Kibana/Logstash is stopped. I mean, I ran command sudo service…
shzetb
  • 13
  • 2
0
votes
0 answers

Flow-Tools - Some Questions

I neet to generate netflow test data for my logstash collector. I've tried a lot of software but I have a lot of problem with everyone. The last im trying is flow-tools under Debian but I have the following problems: timestamp sent is set in…
EviSvil
  • 510
  • 3
  • 21
0
votes
1 answer

Netflow TCP Flags hexidecimal characters not representative of UAPRSF

I'm attempting to perform some statistical analysis of netflow data from a dataset that was provided to me, however I am getting a number of TCP Flags that do not represent the normal UAPRSF format. The following hex values have also been…
Rumpleteaser
  • 4,142
  • 6
  • 39
  • 52
0
votes
1 answer

Using Elasticsearch filter in logstash

I'm trying to use the elastic search filter on logstash for make some Data Enrichment. I got two indexes, and my goal it's get some data from one of them and add it to the other. I configured a logstash filter who search in my elasticsearch and if…
0
votes
0 answers

storing netflow v9 in a time series data base

I am looking for a recommendation about a free linux tool that can collect netflow v9 traffic and store the parsed data in a time series data base for further analysis. I don't need analysis capabilities just good reliable collection and storage…
ncormac
  • 11
  • 3
0
votes
1 answer

Reading Netflow v9 packets in Python

I'm trying to write a collector which listens to incoming Netflow v9 packets on a UDP port and unpacks the records and calculates the total number of bytes. I have a sample collection of data (which I recorded with nfcapd): When I analyse this data…
Edward van Kuik
  • 1,357
  • 1
  • 9
  • 9