Removing a new line feed in json file using Python.

Question

I am in the process of downloading data from firebase, exporting it into a json. After this I am trying to upload it into bigquery but I need to remove the new line feed for big query to accept it.

{   "ConnectionTime": 730669.644775033, 
    "objectId": "eHFvTUNqTR", 
    "CustomName": "Relay Controller", 
    "FirmwareRevision": "FW V1.96", 
    "DeviceID": "F1E4746E-DCEC-495B-AC75-1DFD66527561", 
    "PeripheralType": 9, 
    "updatedAt": "2016-12-13T15:50:41.626Z", 
    "Model": "DF Bluno", 
    "HardwareRevision": "HW V1.7", 
    "Serial": "0123456789", 
    "createdAt": "2016-12-13T15:50:41.626Z", 
    "Manufacturer": "DFRobot"}
{
    "ConnectionTime": 702937.7616419792, 
    "objectId": "uYuT3zgyez", 
    "CustomName": "Relay Controller", 
    "FirmwareRevision": "FW V1.96", 
    "DeviceID": "F1E4746E-DCEC-495B-AC75-1DFD66527561", 
    "PeripheralType": 9, 
    "updatedAt": "2016-12-13T08:08:29.829Z", 
    "Model": "DF Bluno", 
    "HardwareRevision": "HW V1.7", 
    "Serial": "0123456789", 
    "createdAt": "2016-12-13T08:08:29.829Z", 
    "Manufacturer": "DFRobot"}

This is how I need it but can not figure out how to do this besides manually doing it.

{"ConnectionTime": 730669.644775033,"objectId": "eHFvTUNqTR","CustomName": "Relay Controller","FirmwareRevision": "FW V1.96","DeviceID": "F1E4746E-DCEC-495B-AC75-1DFD66527561","PeripheralType": 9,"updatedAt": "2016-12-13T15:50:41.626Z","Model": "DF Bluno","HardwareRevision": "HW V1.7","Serial": "0123456789","createdAt": "2016-12-13T15:50:41.626Z","Manufacturer": "DFRobot"}
{"ConnectionTime": 702937.7616419792, "objectId": "uYuT3zgyez", "CustomName": "Relay Controller", "FirmwareRevision": "FW V1.96", "DeviceID": "F1E4746E-DCEC-495B-AC75-1DFD66527561", "PeripheralType": 9, "updatedAt": "2016-12-13T08:08:29.829Z", "Model": "DF Bluno", "HardwareRevision": "HW V1.7", "Serial": "0123456789", "createdAt": "2016-12-13T08:08:29.829Z", "Manufacturer": "DFRobot"}

I am using python to load the json, read it and then write a new one but can not figure out the right code. Thank you!

here is the outline for my python code

import json
with open('nospacetest.json', 'r') as f:
  data_json=json.load(f)

#b= the file after code for no line breaks is added

with open('testnoline.json', 'w') as outfile:
  json.dump=(b, outfile)

score 4 · Answer 1 · answered Jun 28 '17 at 06:10

4

You only need to make sure that indent=None when you dump you data to json:

with open('testnoline.json', 'w') as outfile:   
    json.dump(data_json, outfile, indent=None)

Quoting from the doc:

If indent is a non-negative integer, then JSON array elements and object members will be pretty-printed with that indent level. An indent level of 0, or negative, will only insert newlines. None (the default) selects the most compact representation.

answered Jun 28 '17 at 06:10

Shai

111,146
38
238
371

Thank you so much but this partially worked. now it does not separate the datasets from each other. Any suggestions? – W. Stephens Jun 28 '17 at 06:19
1

FYI, `indent=None` is the default. – user94559 Jun 28 '17 at 06:24
@smarx of course. it says so explicitly in the doc quoted in the answer. – Shai Jun 28 '17 at 06:25
Right, I'm just not sure why you included `indent=None` in your code. – user94559 Jun 28 '17 at 06:25
@smarx to draw attention to the effect of this parameter. – Shai Jun 28 '17 at 06:26
so the 2 data sets I put in are only part of the json file. there are originally thousands of them I just included them so you knew what I wanted it to look like. The indent=None took away all the new lines but now the datasets are not seperated. I need there to be a new line for every dataset but no new lines inside of the dataset. When I export the dataset and view it on the command line there arent any new lines but when I save it to a json it automatically inserts them. – W. Stephens Jun 28 '17 at 06:29
1

@W.Stephens It's unclear what the format of the file is. Does it start with `[` and end with `]`? And do the dictionaries have commas between them? (What you shared doesn't.) – user94559 Jun 28 '17 at 06:31

score 3 · Accepted Answer · answered Jun 28 '17 at 06:33

3

Reading between the lines, I think the input format might be a single JSON array, and the desired output is newline-separated JSON representations of the elements of that array. If so, this is probably all that's needed:

with open('testnoline.json', 'w') as outfile:
    for obj in data_json:
        outfile.write(json.dumps(obj) + "\n")

answered Jun 28 '17 at 06:33

user94559

59,196
6
103
103

so know this runs but when i go to open the file it cant open it says there is an unknown error. any suggestions? – W. Stephens Jun 28 '17 at 06:45
How are you trying to open the file? – user94559 Jun 28 '17 at 06:47
1

Note that despite the extension `.json`, this will *not* be a valid JSON file. It's a text file with a bunch of JSON objects in it and newlines in between them. So if you're trying to open it in some sort of JSON viewer, that is expected to fail. – user94559 Jun 28 '17 at 06:51

Removing a new line feed in json file using Python.

2 Answers2

Linked