0

I am trying to convert JSON file into data frame in R and had this error when I ran fromJSON in 'jsonlite' package.

Error in fromJSON(content, handler, default.size, depth, allowComments,  : 
  Invalid JSON Node

I figured that the variable order are different in each row. In specific, for example, in the first row, "categories" is the second variable whereas it is the last variable in the third row (Please take a look at the data below). This is how the original JSON file is formatted, and I have several hundred thousand rows like this so can't manually change it.

Anyone can suggest how to fix this and successfully convert the JSON into data frame in R?

[{"asin": "0188399313", "categories": [["Baby"]], "description": "Wee-Go Glass baby bottles by LifeFactory (Babylife) are designed to grow with your child. The included clear cover can also serve as an easy to hold cup. Twist on the solid cap (sold separately) and use your bottles for storing juice or snacks. Perfect for a lunchbox or traveling. The bright colored silicone sleeve (patent pending) helps to protect the bottle from breakage and provides a great gripping surface and tactile experience during feeding. The bottle and sleeve can be boiled or put in the dishwasher together. They can also go in the freezer, making breast milk storage simple.", "title": "Lifefactory 4oz BPA Free Glass Baby Bottles - 4-pack-raspberry and Lilac", "price": 69.99, "imUrl": "http://ecx.images-amazon.com/images/I/41SwthpdD9L._SX300_.jpg", "brand": "Lifefactory", "related": {"also_bought": ["B002SG7K7A", "B003CJSXW8", "B004PW4186", "B002O3JH9Q", "B002O3NLIO", "B004HGSU28"], "also_viewed": ["B003CJSXW8", "B0052QOL1Q", "B004PW4186", "B00EN0OLZ8", "B00EN0OOQY", "B0049YS46K", "B00E64CBLM", "B00F9YOOS6", "B00AH9RPVQ", "B00BCU2R7G", "B002O3NLIO", "B008NZ4X2K", "B005NIDFEW", "B00DKPJCH4", "B00CZNGWWK", "B00DAKJIQ4", "B005CT55IQ", "B0049YRJM0", "B0071IEWD0", "B00E64CA68", "B00IUB3SKK", "B00A7AA6XY", "B001F50FFE", "B002HU9EO4", "B007HP11SQ", "B009WPUMX4", "B002O3JH9Q", "B00F2FT3K6", "B00I5CR35A", "B00BCTY5EK", "B002SG7K7A", "B00F2FLU2U", "B0062ZK0GQ", "B002UOFR66", "B0055LKQQ2", "B00A0FGN8I", "B00HMYCG2W", "B00DHFLUO0", "B0040HMPA2", "B00I5CT9XE", "B008B5MMNO", "B00BQYVNGO", "B00925WM28", "B00BGKC3EY", "B005Q3LSDO", "B0038JDVCE", "B0045I6IA4"], "bought_together": ["B002SG7K7A", "B003CJSXW8"], "buy_after_viewing": ["B003CJSXW8", "B0052QOL1Q", "B004PW4186", "B002SG7K7A"]},},
 {"asin": "0188399518", "categories": [["Baby"]], "description": "The Planet Wise Flannel Wipes are 10 super soft, color coordinated, high-quality wipes that fit perfectly into the Planet Wise Wipe PouchTM.", "title": "Planetwise Flannel Wipes", "price": 15.95, "imUrl": "http://ecx.images-amazon.com/images/I/41otjnA4OGL._SY300_.jpg", "brand": "Planet Wise", "related": {"also_bought": ["B00G96N3YY", "B003XSEV2O", "B000138GNY", "B005WWIE3G", "B005WWI0DA", "B005WWIMGA", "B00DS4WPNK", "B0039VCRPI", "B001QIN6ME", "B004GMGLN8", "B00CJ2OWUG", "B004BDNJW8", "B003N0JXSO", "B003X3R6TO", "B002T5Q01C", "B006J2U4T0", "B00JBJDEC2", "B00380LVLG", "B00GJVM2NW", "B003AJHDQW", "B0043EDGP0", "B0012IJBUE", "B001OI0YWG", "B005DL5970", "B00305GSKS", "B00CQ9UUK8", "B002Y27PQ4", "B00AH8J448", "B00A3JXVZY", "B00483GAJU", "B001J6O6B8", "B0019ID6G2", "B005DL7LGM", "B00APSYDXM", "B004D5KJJA", "B0021HR94K", "0757302661", "B001HX4DNE"], "also_viewed": ["B000138GNY", "B00G96N3YY", "B006J2U4T0", "B00GLBR3C0", "B0039VCRPI", "B0012IJBUE", "B001IA2XGK", "B001R2L6PS", "B003X3R6TO", "B00DS4WPNK", "B005WWIE3G", "B004BDNJW8", "B005WWIMGA", "B005WWI0DA", "B003XSEV2O", "B004A5VGPE", "B004A5TTGC", "B00JBJDEC2", "B004GMGLN8", "B009W6PM9M", "B004RGL9I0", "B0040IWMTK", "B00J3C0XC6", "B00HO1HBC8", "B00G96N18M", "B00A3V0XDA", "B0032AMM9M", "B001QIN6ME", "B00A3V0XEY", "B00CIUAB40", "B00J3C0Y1G", "B005IPE0KI", "B00L0W5B7Y", "B00FZGLKQW", "B00HBYOE3W", "B004A5TT4O", "B00B6KS578", "B002HOQOMA", "B004G792KW", "B002UD6BZS", "B002XVRKHK", "B002HOQOUW", "B002UD6C1Q", "B00APSYDXM", "B00H07AZ42", "B002LZX2BG", "B00BFQVZNO", "B00EZKNVYS", "B00DCLYYJW", "B00AHG8H1Q", "B002QZ64T8", "B003N0JXSO", "B003AJHDQW", "B00DN9NW32", "B00CHPJ0I4", "B005WXMJUY", "B00D4LFC3W"], "bought_together": ["B00G96N3YY", "B003XSEV2O"], "buy_after_viewing": ["B000138GNY", "B006J2U4T0", "B00G96N3YY", "B005WWIMGA"]},},
 {"asin": "0188399399", "description": "The Planet Wise Wipe PouchTM features our patent pending no-leak design so your wipes will stay moist and not wick into your diaper bag. Features a unique snap down design to eliminate wicking from the zipper area. Most standard size wipes will fit into our Wipe PouchTM perfectly! Pouches are made of the same high quality fabrics as our Wet Bags, this pouch will last and last. Pouches are anti-microbial too! After you are done with the baby years, keep this pouch to use for just about anything you need to transport: make-up, toiletries, anything. Made in the USA!", "title": "Planetwise Wipe Pouch", "price": 10.95, "imUrl": "http://ecx.images-amazon.com/images/I/61x8h9u6mxL._SY300_.jpg", "related": {"also_bought": ["B005WWI0DA", "B005WWIMGA", "B006J2U4T0", "B000138GNY", "B003XSEV2O", "B005WWIE3G", "B00DS4WPNK", "B0039VCRPI", "B003GSLCOG", "B0018B15FE", "B002MN3JY2"], "also_viewed": ["B005WWIMGA", "B00G96N3YY", "B00DAI76TC", "B0067GKHVS", "B005WWIE3G", "B004RGL9I0", "B000138GNY", "B00JBJDEC2", "B00CMCJ2AS", "B005WWI0DA", "B002UD6C16", "B00E401FVU", "B004GMGLN8", "B0012IJBUE", "B00BFQVZNO", "B006J2U4T0", "B000VV21G4", "B002HOQOUW", "B003X3R6TO", "B000GZJIVQ", "B0052AIF00", "B005GQ6606", "B003TZHCA4", "B001QIN6ME", "B003XSEV2O", "B00GLBR3C0", "B00DW9R03G", "B00J3C0XC6", "0188399542", "B00DQL6CIE", "B002UD6C1G", "B00A3V0XDA", "B0039VCRPI", "B00EZKNVYS", "B006QRFDHQ", "B00DS4WPNK", "B00CQ9UUK8", "B0087A2PKS", "B00GGNICSW", "B0037NXP18", "B000YCNLSC", "B00FJG945W", "B00FZGLKQW", "B004A5VGPE", "B00D4LFC3W", "B00G96N18M", "B00439UNR4", "B004A5TT4O", "B005DL7LGM", "B006SD2JUW", "B006Z6HXYY", "B006QRFDZI", "B00I0P7VYK", "B00AZWDM0I", "B000ZKHVMU", "B007799PGC", "B00H58UNMU", "B00JKUU3TE"], "buy_after_viewing": ["B005WWIMGA", "B00G96N3YY", "B00DAI76TC", "B0067GKHVS"]},, "categories": [["Baby"]]},
 {"asin": "0316967297", "description": "Hand crafted set includes 1 full quilt (76x86 inches) and 2 standard shams (20x26 inches). Face cloth and fill are 100 natural cotton. Prewashed for out of the bag comfort. Hand crafted with embroidery. Machine washable. Made in China", "title": "Annas Dream Full Quilt with 2 Shams", "price": 109.95, "imUrl": "http://ecx.images-amazon.com/images/I/51%2BZ1%2BNeukL._SY300_.jpg", "related": {"also_viewed": ["B009LTER3W", "B00575TI5Q", "B004NSYYJI"], "buy_after_viewing": ["B009LTER3W", "B001MX5EE6", "B00575TI5Q", "B0029009TG"]},, "categories": [["Baby"]]},
 {"asin": "0615447279", "categories": [["Baby"]], "description": "Thumbuddy To Love- The Binky Fairy helps children give up pacifier sucking without the fuss and tears. The adorable book comes with a matching Binky Fairy puppet and just like the Tooth Fairy, the book is read the night before so children understand where thier pacifiers will go. They awake to find an adorable Binky Fairy puppet under thier pillow knowing  that the Binky Fairy came! Each book comes with a success chart and stickers. Winner of  the PTPA Awards (Parent Tested, Parent Approved). Recommended for ages 2-4.", "title": "Stop Pacifier Sucking without tears with Thumbuddy To Love s Binky Fairy Puppet and Adorable Book", "price": 16.95, "imUrl": "http://ecx.images-amazon.com/images/I/51RKKENlq%2BL._SY300_.jpg", "brand": "", "related": {"also_bought": ["0979670004", "1601310234", "B005G172KE", "1575422573", "1905417896", "B0044D0HA2", "B001GQ2CPI"], "also_viewed": ["0979670004", "1575422573", "1493535943", "0615273645", "1601310234", "0992616727", "0615273793", "0763623644", "1907152962", "0992616719", "B0016ZX7AI", "0698400488", "1608442713", "0451416058", "1581176848", "B0072CTECY", "B005G172KE", "B0016P8K3W", "0375822704", "0979201004", "0961478020", "160131048X", "1572245859"], "bought_together": ["B005G172KE"], "buy_after_viewing": ["0979670004", "1575422573", "B005G172KE", "1493535943"]},}]
Emily
  • 305
  • 3
  • 13
  • Variable order shouldn't matter. Have you tried validating your JSON file with something like https://jsonlint.com/ to make sure ti's a valid JSON file? – MrFlick Jul 17 '17 at 20:01
  • Yeah, these aren't valid JSON files. You have extra commas where they are not allowed. You should fix whatever process is generating these invalid files. – MrFlick Jul 17 '17 at 20:04
  • @MrFlick, Thanks MrFlick. I deleted the unnecessary comma and ran the same command, and it returned only the first row of the the data correctly (i.e., asin: 0188399313). 'jsonlite' in R is not a good tool for parsing this type of JSON file? Somebody have any idea how to parse this? I have subsequent experience in R but have zero experience in parsing JSON file. It is quite confusing. – Emily Jul 18 '17 at 01:24
  • Did the file pass the validator after removing that comma? It looked like you had quite a few formatting problems in that file. That means it's just not a true JSON file, just a file that kind of looks like a JSON file. Since that's not a standard format you are kind of on your own to parse it. – MrFlick Jul 18 '17 at 01:26
  • @MrFlick, I tried json validator and it gave this error; Error: Parse error on line 19: ...S\n* 1 Belt Clip"} { "asin": "01104004 ----------------------^ Expecting 'EOF', '}', ',', ']', got '{' Maybe it is not formatted correctly but do not understand exactly what the error means. – Emily Jul 18 '17 at 01:42
  • The original JSON file is formatted named 'strict json', which I don't know exactly what it is. – Emily Jul 18 '17 at 01:52

0 Answers0