D3.js join data from multiple csv files

Question

I am new to D3 and have been tasked with creating a dash board for displaying oil well data. The data is in 3 csv files with the following headings.

Company Data: CompanyName,FacilityName,WellName,WellId,WellGasNRI,WellOilNRI,WellShrinkageFactor,WellYieldFactor
Well Budget Estimates: WellId,Month,Year,DaysInMonth,GrossOil,GrossGas,GrossBOE,NetBOE
Actual well production: WellId,Date,GrossOil,GrossGas,NetOil,NetGas,GasWithShrinkage,GasWithYield

I have worked with d3.nest() to groups records within each file but would like to merge the grouped arrays into a single structure.

One of the main displays is to compare budget estimate to real production by day for each well then by facility.

Any help would be greatly appreciated.

score 3 · Answer 1 · edited May 23 '17 at 10:29

One way to do it is to nest several d3.csv calls, as explained here: Importing data from multiple csv files in D3:

d3.csv("file1.csv", function(data1) {
  d3.csv("file2.csv", function(data2) {
     d3.csv("file3.csv", function(data3) {
        // do something with the data
        console.log("CSV1", data1);
        console.log("CSV2", data2);
        console.log("CSV3", data3);
     });
  });
});

Another option (which I prefer) is to use d3.queue (which is a plugin - you need to import queue.js separately):

queue()
    .defer(d3.csv, 'file1.csv')
    .defer(d3.csv, 'file2.csv')
    .defer(d3.csv, 'file3.csv')
    .await(processData);

function processData(data1, data2, data3) {
    // do something with the data
    console.log("CSV1", data1);
    console.log("CSV2", data2);
    console.log("CSV3", data3);
}

From there you can use d3 and JavaScript array manipulation functions to combine, merge, split and do anything you want with the data objects. For example, you might want to use d3.nest().key() to organize your CSVs so they can be selected using the WellId field and then combine them using nested for loops or some other mechanism. Here are some good tutorials and references concerning data manipulation in D3:

D3.js join data from multiple csv files

1 Answers1