Rrd4j not able to receive samples from MongoDB

Question

I have integrated rrd4j (3.1) with mongodb (3.2). But the sample data doesn't seem to be persistent after closing RrdDb and then re initializing the object. I see the binary data getting updated in mongo db when the rrdDb.close() is called. And on open the data is being queried from mongodb and the binary data is getting updated to byte buffer. But on dumping the data after the reconnect, all the sample data is is replace by NaN. Could someone please help me on this.?

Adding the code for RrdDb intialising, Mongo Ractory creation and the dump outputs before close and after reconnect.

RrdDb intialization

RrdDef rrdDef = new RrdDef(rrdPath,(System.currentTimeMillis() - TimeUnit.DAYS.toMillis(300))/1000,10);
rrdDef.addDatasource(dsName, DsType.GAUGE ,  20, 1d, Double.MAX_VALUE);

int secondUnitStep = (int)(TimeUnit.MINUTES.toSeconds(1)/10);
int secondUnitRows = (int)(TimeUnit.HOURS.toSeconds(1)/TimeUnit.MINUTES.toSeconds(1));

rrdDef.addArchive(ConsolFun.TOTAL, 0.5, 6, 60); 

RrdDb rrdDB = new RrdDb(rrdDef,rrdMongoFactory);

Rrd Mongo Factory class

package mx.july.jmx.proximity.util;

public class RrdMongoFactory extends RrdBackendFactory {

    private String name;
    private final DBCollection rrdCollection;

    public RrdMongoFactory(String name, DBCollection rrdCollection) {
        this.name = name;
        this.rrdCollection = rrdCollection;
        this.rrdCollection.createIndex(new BasicDBObject("path", 1), "path_idx");
        RrdBackendFactory.registerAndSetAsDefaultFactory(this);
    }

   public void setName(String name) {
       this.name = name;
   }


    public String getName() {
        return name;
    }

    /** {@inheritDoc} */
    @Override
    protected RrdBackend open(String path, boolean readOnly) throws IOException {
        return new RrdMongoDBBackend(path, rrdCollection);
    }



    /** {@inheritDoc} */
    @Override
    protected boolean exists(String path) throws IOException {
        BasicDBObject query = new BasicDBObject();
        query.put("path", path);
        return rrdCollection.findOne(query) != null;
    }

    /** {@inheritDoc} */
    @Override
    protected boolean shouldValidateHeader(String path) throws IOException {
        return false;
    }




}

RrdDb dump before close

== HEADER ==
signature:RRD4J, version 0.2 lastUpdateTime:1488536144 step:10 dsCount:1 arcCount:3
== DATASOURCE ==
DS:320:GAUGE:20:1.0:1.7976931348623157E308
lastValue:8.0 nanSeconds:0 accumValue:32.0
== ARCHIVE ==
RRA:TOTAL:0.5:6:60
interval [1488532560, 1488536100]
accumValue:117.0 nanSteps:0
Robin 5/60: NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN +1.7860000000E02 NaN +1.7130000000E02 +1.6600000000E02

RrdDb dump after reconnect

== HEADER ==
signature:RRD4J, version 0.2 lastUpdateTime:1462616268 step:10 dsCount:1 arcCount:3
== DATASOURCE ==
DS:320:GAUGE:20:1.0:1.7976931348623157E308
lastValue:NaN nanSeconds:8 accumValue:0.0
== ARCHIVE ==
RRA:TOTAL:0.5:6:60
interval [1462612680, 1462616220]
accumValue:NaN nanSteps:4
Robin 0/60: NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN

score 0 · Accepted Answer · answered Mar 13 '17 at 11:40

Got the solution for this from another group. Thought of posting this here too.

So basically for each run you need to open the rrdDb connection at the satrt and close it at the end of the run.

When creating RrdDb object for the very first time, you need to define the RrdDef and the back-end factory.(code snippet for the same posted earlier). At the end of the run when the data is collected, you need to close the RrdDb connection (rrdDb.close()). Only then the data will be written to the backend (in this case mongo db). On the next open rather than defining the rrdDef just pass the rrdPath as an attribute to rrdDb object, syntax for the same is mentioned below, using which mongo make a query and retrieves the corresponding information.

rrdDB = new RrdDb(rrdPath, rrdMongoShardFactory);

Please note if there is no entry for the path in mongo, then the above command will throw a file not found exception. Sample snippet given below

 public RRDInfo openRrdDB() throws IOException {
    try{
        rrdDB = new RrdDb(rrdPath, rrdMongoShardFactory);
    }catch(FileNotFoundException e) {
        rrdDef = new RrdDef(rrdPath,(System.currentTimeMillis() - TimeUnit.DAYS.toMillis(300))/1000,Configurator.RRD_STEP_INTERVAL);
        rrdDef.addDatasource(dsName, DsType.GAUGE ,  Configurator.RRD_STEP_INTERVAL*2, 1d, Double.MAX_VALUE);

        int secondUnitStep = (int)(TimeUnit.MINUTES.toSeconds(1)/Configurator.RRD_STEP_INTERVAL);
        int secondUnitRows = (int)(TimeUnit.HOURS.toSeconds(1)/TimeUnit.MINUTES.toSeconds(1));

        int hourUnitStep = (int)(TimeUnit.HOURS.toSeconds(1)/Configurator.RRD_STEP_INTERVAL);
        int hourUnitRows = (int)(TimeUnit.DAYS.toSeconds(2)/TimeUnit.HOURS.toSeconds(1));

        int dayUnitStep = (int)(TimeUnit.DAYS.toSeconds(1)/Configurator.RRD_STEP_INTERVAL);
        int dayUnitRows = (int)(TimeUnit.DAYS.toSeconds(365)/TimeUnit.DAYS.toSeconds(1));

        rrdDef.addArchive(ConsolFun.TOTAL, 0.5, secondUnitStep, secondUnitRows); 
        rrdDef.addArchive(ConsolFun.TOTAL, 0.5, hourUnitStep, hourUnitRows); 
        rrdDef.addArchive(ConsolFun.TOTAL, 0.5, dayUnitStep, dayUnitRows);
        rrdDB = new RrdDb(rrdDef,rrdMongoShardFactory);
    }
    return this;
}

Rrd4j not able to receive samples from MongoDB

1 Answers1