Parsing uneven json as a tibble

Question

How do you converting an uneven json file into tidy data? Here I have an uneven json file, and I'd like to change into tidy data.

library(tidyverse)
library(jsonlite)

read_json("https://api.osf.io/v2/search/?q=*&page=100&format=jsonapi") %>% 
  as_tibble()
#> Error: Tibble columns must have compatible sizes.
#> * Size 5: Column `links`.
#> * Size 6: Column `search_fields`.
#> * Size 10: Column `data`.
#> ℹ Only values of size one are recycled.

There are other questions about lists of unequal length, I think the common element is using `\`lengths<-\`` to extend all lists to the same length. — r2evans, Jul 31 '20 at 22:34
If you can handle nested data, then why not leave it as a list? Why are you trying to force it into a rectangular tibble? It's unclear how one would turn that data into rows and columns just looking at it. You should provide the desired output for that sample input so that possible solutions can be tested and verified. — MrFlick, Jul 31 '20 at 22:36
@MrFlick If I can get one row / page I think I could work with that. Working with discrete lists will be too challenging — John-Henry, Jul 31 '20 at 22:40
Sorry I don't have an desired output, I'm not sure how to approach this question otherwise — John-Henry, Jul 31 '20 at 22:41

Oliver · Answer 1 · 2020-07-31T23:08:51.430

It is hard to come up with a concrete answer to a question so open. The data is also heavily nested, and it is not clear, how you want to unnest this properly. A place to start however is to unnest_wider each of the elements in your list, and then try to figure where you want to go from there

library(tidyverse)
library(jsonlite)

data <- read_json("https://api.osf.io/v2/search/?q=*&page=100&format=jsonapi")
unnest_wider(d = tibble::tibble(data[[1]]), c(d))
# The second element seems to be doubly nested
unnest_wider(unnest_wider(d = tibble::tibble(data[[2]]), c(d)), c(related))
unnest_wider(d = tibble::tibble(data[[3]]), c(d))
unnest_wider(d = tibble::tibble(data[[4]]), c(d))

From there you can merge, unnest certain columns further and so forth, but without more information it is hard to come with anything more concrete.

Thanks for the response! However I'm getting `Error in UseMethod("tbl_vars") : no applicable method for 'tbl_vars' applied to an object of class "list"` error — John-Henry, Jul 31 '20 at 22:54
Right, i forgot to wrap `data[[i]]` in `tibble::tibble(..)`` I'll update it quickly. — Oliver, Jul 31 '20 at 23:08

Parsing uneven json as a tibble

1 Answers1