1

I have the following local Json file (around 90MB):

For my data to be accessible, I want to create smaller JSON files that include exactly the same data but only 100 of the array entries in Readings.SensorData every time. So a file that includes the first 100 readings, then a file that includes readings 101-200, and so on... I am aware about the ijson library but I cannot figure out to do this in the most memory effective way.

Edit: Just to note I know how to do this with the standard JSON library but because it is a big file I want to be able to do this in a way that doesn't come to a complete halt.

halfer
  • 19,824
  • 17
  • 99
  • 186
Mohammad
  • 7,344
  • 15
  • 48
  • 76

1 Answers1

0

Use dict.items() in python

You will need json, package in python

Use json.loads() features and it will result in a python dictionary.

Nirjal Paudel
  • 201
  • 5
  • 11
  • Thanks I'm looking to avoid using the standard library as this is a large amount of data and loading it all would cause the system to be slow. – Mohammad Apr 23 '20 at 08:26