How do I import data from json format into R using jsonlite package

Question

I am doing this

newzips=fromJSON("http://media.mongodb.org/zips.json")

You can see the data yourself at http://media.mongodb.org/zips.json

and getting thus

str(newzips)
 List of 5
 $ city : chr "ACMAR"
 $ loc  : num [1:2] -86.5 33.6
 $ pop  : num 6055
 $ state: chr "AL"
 $ _id  : chr "35004\"}{\"city\":\"ADAMSVILLE\",\"loc\":[-86.959727,33.588437],\"pop\":10616,\"state\":\"AL\",\"_

Think you have to read it and convert line by line, or join it with commas and convert it... — Spacedman, Jul 01 '14 at 16:00
Don't really understand the downvotes here - question is reproducible, output is given etc. Reasonable to expect a .json file is truly JSON... — Spacedman, Jul 01 '14 at 16:15

Jeroen Ooms · Accepted Answer · 2015-03-28T20:42:17.643

7

This format is called jsonlines. You can import it using the stream_in function in jsonite:

library(jsonlite)
zips <- stream_in(url("http://media.mongodb.org/zips.json"))

If the server uses https, you can use the curl package:

library(jsonlite)
library(curl)
zips <- stream_in(curl("https://media.mongodb.org/zips.json"))

Datasets where each line is a record are usually nosql database dumps. Because they might be too large to parse all at once, they are meant to be imported line-by-line, which is exactly what jsonlite does.

edited Mar 28 '15 at 20:42

answered Dec 07 '14 at 00:47

Jeroen Ooms

31,998
35
134
207

Absolutely. I just streamed it with ease. Took about 1.7 seconds – Rich Scriven Dec 07 '14 at 01:36

score 4 · Answer 2 · answered Jul 01 '14 at 16:04

readLines + mush it into a JS array with some brackets and comma-separation:

> json = fromJSON(paste("[",paste(readLines("http://media.mongodb.org/zips.json"),collapse=","),"]"))
Warning message:
In readLines("http://media.mongodb.org/zips.json") :
  incomplete final line found on 'http://media.mongodb.org/zips.json'
> head(json)
        city                 loc   pop state   _id
1      ACMAR -86.51557, 33.58413  6055    AL 35004
2 ADAMSVILLE -86.95973, 33.58844 10616    AL 35005
3      ADGER -87.16746, 33.43428  3205    AL 35006
4   KEYSTONE -86.81286, 33.23687 14218    AL 35007
5   NEW SITE -85.95109, 32.94145 19942    AL 35010
6     ALPINE -86.20893, 33.33116  3062    AL 35014

How do I import data from json format into R using jsonlite package

2 Answers2