0

I am starting to study Node JS and I am trying to convert an external JSON file to CSV format.

The operation that I am trying to achieve is the following: in an internal file I have two urls that lead to an external JSON each (api1 and api2), given the url http://localhost:3000/?api=api1, you have to download the JSON of the api1 in CSV format with a maximum of 50 lines.

This is the code I have so far (I have added some modules that I have seen are necessary):

import { Request, Response } from 'express';

const converter = require("json-2-csv");
const fetch = require("node-fetch");
const fs = require("fs");
const flatten = require('flat');

const conf = require(`../conf/${process.env.NODE_ENV}.json`);
const maxRecords = 50;

class IndexController {
  public async index(req: Request, res: Response) {
    const api =req.query.api; //api1
    const url = conf.API_MOCS[`${api}`].url; //https://mock.com/api1

    const csvWriter = createCsvWriter({
    path: 'path/to/file.csv',
    header: [
        {id: 'index', title: 'Index'},
        {id: 'index_start_at', title: 'Index start'},
        {id: 'integer', title: 'Integer'},
        {id: 'float', title: 'Float'},
        {id: 'name', title: 'Name'}
    ]
});

fetch(url)
  .then(res => res.json())
  .then(json => csvWriter.writeRecords(json.myItems));
  }
}
export const indexController = new IndexController();

This is my internal file that contains the url of the JSON:

{
  "API_MOCS": {
    "api1": {
      "url": "https://mock.com/api1"
    },
    "api2": {
      "url": "https://mock.com/api2"
    }
  }
}
six
  • 87
  • 1
  • 1
  • 7

1 Answers1

1

You should first ‘fetch’ the json from the URL. There are plenty of libraries available to help you with this. Take a look at: https://www.npmjs.com/package/node-fetch Or https://www.npmjs.com/package/axios

Then you can write the CSV file. I’d recommend using a library for that too. First hit: https://www.npmjs.com/package/csv-writer

import { Request, Response } from 'express';

const fetch = require("node-fetch");
const createCsvWriter = require('csv-writer').createObjectCsvWriter;

const conf = require(`../conf/${process.env.NODE_ENV}.json`);
const maxRecords = 50;

class IndexController {
  public async index(req: Request, res: Response) {
    const api =req.query.api; //api1
    const url = conf.API_MOCS[`${api}`].url; //https://mock.com/api1
    const csvWriter = createCsvWriter({
      path: 'path/to/file.csv',
      header: [
        {id: 'name', title: 'NAME'},
        {id: 'lang', title: 'LANGUAGE'}
      ]
    });

    fetch(url)
      .then(res => res.json())
      .then(json => csvWriter.writeRecords(json.myItems));
  }
}
export const indexController = new IndexController();
Vincent Bitter
  • 1,000
  • 1
  • 6
  • 15
  • Yes, I have installed the node-fetch and fs modules as I put in the question. My problem is that I don't know how to follow, how can I read the JSON and download it in CSV format? – six Jun 06 '21 at 10:24
  • I’ve added an example for you – Vincent Bitter Jun 06 '21 at 10:36
  • It shows me the following error: ``TypeError: Cannot convert undefined or null to object``. I have updated the question – six Jun 06 '21 at 10:43
  • 1
    +1 for suggesting a fully-debugged library for CSV handling. CSV looks simple, but it has many little quirks, and there's not much value to be had from rediscovering them. – O. Jones Jun 06 '21 at 10:55
  • Are you getting this TypeError in the csv-writer module or before that? It is hard to reproduce without having an example output of the API's. Could you provide that? – Vincent Bitter Jun 06 '21 at 11:48
  • I have updated with the current code. I am testing with the ``json-2-csv`` module and I have created a method to read the information from the JSON but I cannot download it in CSV – six Jun 06 '21 at 12:28
  • I have tried with createCsvWriter but it shows me two errors in fetch: Parameter 'res' implicitly has an 'any' type and Parameter 'json' implicitly has an 'any' type – six Jun 06 '21 at 12:32