Convert JSON LINES file to JSON format?

Question

I have file (around 6 GB) that each line is JSON.

{"name":"name1", "age":40, "car":null}
{"name":"name2", "age":30, "car":null}
{"name":"name3", "age":30, "car":null}

How can I convert it into a JSON array with Python?

create a list, then iterate over each line of your file and append it to your list, finally export the list as json — , Aug 09 '21 at 11:40
You need to enclose the file's contents in `[` and `]` brackets and add a comma between each line — there's no need to read the entire thing into memory at once to this. — martineau, Aug 09 '21 at 11:49

score 1 · Accepted Answer · answered Aug 09 '21 at 12:15

1

import fileinput
for line in fileinput.input("test.txt", inplace=True):
    if 1 != fileinput.filelineno():
        print(',{}'.format(line), end='')
    else:
        print('[{}'.format(line), end='')
open("test.txt","a").write(']')

answered Aug 09 '21 at 12:15

MicrosoctCprog

460
1
3
23

score 0 · Answer 2 · answered Aug 09 '21 at 11:50

0

Load each line and save it into a list.

with open('file.txt', 'r') as in_file:
    lines = [json.loads(line) for line in in_file.readlines()]

Then you can save that to a file like with json.dump()

with open('out.json', 'w') as out_file:
    json.dump(lines, out_file)

answered Aug 09 '21 at 11:50

Fynngin

9
3

1

No need to use `readlines()`: `for line in in_file:` would work just as well and use half as much memory. – martineau Aug 09 '21 at 11:52
1

Actually you don't even need to do that — just do `json.dump(list(in_file), outfile)`. – martineau Aug 09 '21 at 11:58

Convert JSON LINES file to JSON format?

2 Answers2