0

My API call returns an ndjson format that wont allow me to use "data = response.json". My only work around is doing this. This give me a string format that is difficult to parse. After put into a pandas df, it is about 20 columns of dictionaries of dictionaries. Is there a better way of getting the ndjson data and/or parsing these columns?

import pandas as pd 
from io import StringIO
import ndjson as nd

data = response.text()
df = pd.read_json(StringIO(data), lines = True)
user17629522
  • 105
  • 8
  • What do you mean 'a strange format that is not technically ndjson'. You've got an ndjson string which you are processing as if it were a file handle with StringIO. It doesn't really seem like there's anything wrong with that. – Nick Bailey Mar 08 '22 at 19:27
  • Hi Nick, I updated my question to make more sense. Does StringIO() just convert the ndjson data into a large string? Will this have any affect on parsing the json to convert to csv? – user17629522 Mar 08 '22 at 19:32
  • `data` above is already a large string, StringIO just wraps that string in a line-based reader interface similar to the interface for files. – Nick Bailey Mar 08 '22 at 19:36
  • Then from there i converted into a pandas df. After that, the columns are a bunch of either dicts of dicts or lists. – user17629522 Mar 08 '22 at 19:39

0 Answers0