1

I converted the Glove 840B text file into JSON vectors in the format of {"word": [300 Dimensional feature vector]}, and I need to query this JSON file thousands of times in my program to get vectors for thousands of words.

I was thinking of storing this JSON file in an alternative format, like a SQL database, but I'm not sure.

Any help would be appreciated, thanks!

kevdoge
  • 11
  • 1
  • 1
  • storing whole 13GB of a json file in sql table might not be the best approach here, since sql also needs to create other relations between data, the actual storage requirements will be well above 13GB, i would suggest looking into streaming JSON parsers like ijson and json-streamer – syfluqs Mar 01 '20 at 00:19

0 Answers0