0

I am trying to convert csv to json. This CSV contains emojis, i guess that might be the problem.

csvtojson or any other module or even any custom method is returning unknown characters/garbage characters. My csv file link below:

CSV

The code i am using is:

let csvFilePath = path.join(\__dirname, 'reports/xyz.csv');

let str1 = fs.readFileSync(csvFilePath, { encoding: 'utf8' }).toString();

csv({noheader:true,output: "csv"}).fromString(str1).then((csvRow)=\>{console.log(csvRow)});

It is returning a bunch of \x00c\x00o\x00m\x00.\x00b\x00h\x00a\x00k\x00t\x00 characters.

PRATHEESH PC
  • 1,461
  • 1
  • 3
  • 15
Hiten Khatri
  • 1
  • Why are you reading the file first into a string, instead of using [`csv.fromFile()`](https://www.npmjs.com/package/csvtojson#from-csv-file-to-json-array)? – Geshode Jun 13 '23 at 09:20
  • I have also used csv.fromFile() that is resulting in same problem. – Hiten Khatri Jun 13 '23 at 10:02

1 Answers1

0

I have tried to print your csv file in console and it indeed shows unicode character identifier.

The problem is not with csvtojson module rather with the csv.

I have opened webstorm debugger while running your code. This is the first line of your csv as json object. See the vertical lines between each character both in every key and value of the row object. This is a problem with the format of the csv and the software used to write it. The csv you provided has invisible characters. I don't actually know why this would be present in all of the characters but it does exist in the file.

Webstorm debugger screenshot

Now for your options:

  1. You can sanitize the csv first. Get rid of this invisible character and then print/parse,
  2. Regenerate the csv if it was autogenerated. If not, try copy pasting to another csv by copying the tabular data from the google spreadsheet app. I don't know if this will work in this case. Be sure to paste value only, not formatting which might be helpful.
Anup K. Tarafder
  • 11
  • 6