1

I need to iterate through all .rrd files inside a given directory and do fetch data inside each rrd database and do some actions and export them into single csv file in python script!

How can this be done in a efficient way? it's ok to advice me how to loop and access database data over multiple files?

Swe Zin Phyoe
  • 149
  • 1
  • 2
  • 11
  • You ask at least two questions here: How to loop over files (google "glob") and how to read and process `.rrd` files (google "rrd python"). Please do your research and then ask for one thing at a time. – j08lue Feb 13 '18 at 08:08
  • `glob` is usually not a good way to iterate over files within python. – Arne Feb 13 '18 at 08:09
  • i m ok with the fetching data from rrd files and export into csv file. The problem is there are multiple rrd files and i want to export them all in one single csv file. so i dun know how to loop every single rrd file and access into each single rrd files! – Swe Zin Phyoe Feb 13 '18 at 09:38

1 Answers1

1

I assume you have a rrdtool installation with python bindings already on your system. If not, here is an installation description.

Then, to loop over .rrd files in a given directory and performing a fetch:

import os
import rrdtool

target_directory = "some/directory/with/rrdfiles"
rrd_files = [os.path.join(target_directory, f) for f in os.listdir(target_directory) if f.endswith('.rrd')]
data_container = []
for rrd_file in rrd_files:
    data = rrdtool.fetch(rrd_file, 'AVERAGE'  # or whichever CF you need
        '--resolution', '200',                # for example
    )                                         # and any other settings you need
    data_container.append(data)

The parameter list follows rrdfetch.

Once you have whatever data you need inside the rrd_files loop, you should accumulate it in a list of lists, with each sublist being one row of data. Writing them to a csv is as easy as this:

import csv

# read the .rrd data as above into a 'data_container' variable

with open('my_data.csv', 'w', newline='') as csv_file:
    rrd_writer = csv.writer(csv_file)
    for row in data_container:
        rrd_writer.writerow(row)

This should outline the general steps you have to follow, you will probably need to adapt them (the rrdfetch in particular) to your case.

Arne
  • 17,706
  • 5
  • 83
  • 99