Is there an npm package that will convert unstructured data to csv?

Question

Trying to export some data into csv in Javascript. Trying to find a npm module to do this... however - the data I have could have different headers per row...

for example my data could be this:

[ { name: 'John', color: 'Blue', age: 25 }, { name: 'Ursula', color: 'Red', food: 'pasta'},...]

Ideally, the csv would export to this:

name, color, age, food
John, Blue, 25,
Ursula, Red,,pasta

I've looked into fast-csv and it doesn't seem to entertain dynamic headers (the above stated needs)... (in the above example, fast-csv would have static headers provided by the first entry only (only name,color and age))

This data isn't really "unstructured", it just has some optional fields. — ADyson, Apr 27 '19 at 21:57
Not against changing the title using a different word than "unstructured", however - I wouldn't consider the additional fields "optional" either. — deblearns1, Apr 27 '19 at 21:59
Well some of them are not always filled in for any given record, so clearly there must be scope for whoever entered the data not to complete every field. Or are you saying they can arbitrarily declare any properties they like, so the next record could contain "jeans : blue" or something else completely unpredictable? Is there a finite list of possible fields, or not? — ADyson, Apr 27 '19 at 22:02
@ADyson, yes - the next record could contain "jeans: blue". It's possible - each record is completely unpredictable. — deblearns1, Apr 27 '19 at 22:11

score 1 · Accepted Answer · answered Apr 27 '19 at 21:52

Here's some code that will generate that output based on a similar data structure with different key names in the objects.

const data = [{ name: 'John', color: 'Blue', age: 25 }, { name: 'Ursula', color: 'Red', food: 'pasta'}, { size: 12, feet: 'small', name: 'Bob'}];

function getHeadings(data) {

  // `reduces` over the array combining all the
  // different object keys into one array
  // removing the duplicates by converting it to a
  // Set, and then converting it back to an array again
  return [...new Set(data.reduce((acc, c) => {
    const keys = Object.keys(c);
    return acc.concat(keys);
  }, []))];
}

const headings = getHeadings(data);

// Iterate over the data...
const out = data.reduce((acc, c) => {

  // ...grab the keys in each object
  const keys = Object.keys(c);

  // ...create a new array filled with commas
  const arr = Array(headings.length).fill(',');

  // For each key...
  keys.forEach(key => {

    // ...find its index in the headings array
    const index = headings.findIndex(el => el === key);

    // ...and replace the comma in the array with the key
    arr[index] = `${c[key]},`;
  });
 
  // Add the array to the output
  return acc.concat(arr.join(''));
}, []).join('\n');

console.log(`${headings}\n${out}`);

Is there an npm package that will convert unstructured data to csv?

1 Answers1