0

Hello guys I have a assignment, which I have honestly no idea how to complete. I have a huuuge Json file which is 22 GB big, so I have to get the data from it but it's full with empty spaces and new lines and I have to trimm does before even opening the file. I tried getting the data and trim it but it says it is out of memory I even set the memory limit to -1 if anyone has any idea how is that supposed to happen please I am open to suggestions.

Stiliyan
  • 176
  • 11
  • have you tired chunking your trimming/reading process? – madalinivascu Oct 12 '17 at 11:35
  • Look for a *JSON stream parser*, allowing you to read the file bit by bit and not in one chunk. – deceze Oct 12 '17 at 11:38
  • Post your code, errors, what you have tried. We will not write the code for you. That being said, @deceze beat me to it, you have to parse the file, not try to load it and process after. – Nic3500 Oct 12 '17 at 11:38
  • So I guess the way is to read the JSON file line by line any idea how can I get each line without reading the whole file ? – Stiliyan Oct 12 '17 at 19:29

1 Answers1

3

There is simple and cool parser for parsing large JSON files.

https://github.com/salsify/jsonstreamingparser

Also maybe you will find an answer on the following links
Parse large JSON file
Processing large JSON files in PHP

JohnWayne
  • 651
  • 9
  • 28