0

I am looking for a recommendation about a free linux tool that can collect netflow v9 traffic and store the parsed data in a time series data base for further analysis. I don't need analysis capabilities just good reliable collection and storage performance

ncormac
  • 11
  • 3
  • Be more specific in your requirements: which timeseries DB for storage, expected flow data scale, how close to realtime you want data to be stored in DB, etc. `pmacct` is a proven collector. Can store data in mysql/postgres, send data to kafka or produce csv files, which you can load to tsdb of your choice, Also have a look at Ntopng, FastNetMon. Clickhouse DB is good for large scale storage, VictoriaMetrics or InfluxDB are alternatives. There are many other options for collectors and DBs. – Yuri Lachin Jun 26 '20 at 14:46
  • Thank you very much. I'm interested in collecting around 100K flows per sec (big commercial routers). I don't have to store the data for long, something like 10m before flushing. I thought about using influxdb – ncormac Jul 01 '20 at 13:40
  • 1
    Keep in mind, that InfluxDB doesn't like high cardinality series. Detailed flow aggregation scheme with keys like source_ip+destination_ip can easily cause troubles. – Yuri Lachin Jul 02 '20 at 10:38
  • If you don't need analytics at all you can just use kafka as a reliable short-term storage. – Yuri Lachin Jul 02 '20 at 10:40
  • A bit late, but maybe check out https://github.com/paihu/netflow_exporter, a NetFlow collector and Prometheus exporter. – TLin Mar 09 '21 at 01:04

0 Answers0