How do I iterate over a nested list, where nodes may be missing

Question

I am constructing a data frame from a (very deeply nested) json source.

I have successfully mapped the list elements I want to extract, and can almost! construct the tibble I want.

Except that some of the list elements are missing, if there is no data. Like so (I want the chr at position "five", and NA if it is missing, like in the second list):

nested_list  <- list(
    list(
        x = list(one = "this"),
        y = list(two = list(
                     three = list(
                           four = data_frame(
                                five = "is"))))),
   list(
       x = list(one = "a"),
       y = list(two = list(three = list()))))

How can I supply a fallback if the outer elements are missing? I'm fumbling around with map_if, but can't figure out a predicate function to test for presence of an element. I get

Error: Predicate functions must return a single `TRUE` or `FALSE`, not NULL

so far.

I think I have a reproducible example:

library("tidyverse")

nested_list  <- list(
    list(
        x = list(one = "this"),
        y = list(two = list(
                       three = list(
                               four = data_frame(
                                      five = "is"))))),
   list(
       x = list(one = "a"),
       y = list(two = list(three = list()))))

test_df <- nested_list %>%
    map(~ .x) %>%
    tibble(
        one = map_chr(., c("x", "one")),
        three = map_chr(., c("y", "two", "three", "four", "five"))
   )

this returns

Error: Result 2 must be a single string, not NULL of length 0

Where I'd like to just have a missing value in my final tibble.

reducing the tibble constructor to

test_df <- nested_list %>%
    map(~ .x) %>%
    tibble(
        one = map_chr(., c("x", "one")),
        three = map(., c("y", "two", "three"))
   )

gives me NULL values in the second row of the data frame; and, as expected, a nested list in the first row. `map_df(., c("one","two", "three", "four")) gives me uneven columns, and the tibble construction fails.

And my google fu is failing me.

you need to make this reproducible. please include a sample of your data, what you have tried and that the desired output is. It is very hard to help you with so little information — morgan121, Nov 15 '19 at 05:57
yes. sorry about that - I've attempted to produce an example, but seem to be flailing about a bit in reproducing the wonky list in question. — Martin Andersen, Nov 16 '19 at 04:23
https://drive.google.com/open?id=1krPtBND-anusscZ5fRT3M5SwP8MMlO49 for my data file to pass in as jsondata — Martin Andersen, Nov 16 '19 at 04:40

score 0 · Accepted Answer · edited Nov 22 '19 at 13:46

0

As I suspected, I was holding it wrong. There is a .null parameter to map, for just this purpose.

So:

test_df <- nested_list %>%
map(~ .x) %>%
tibble(
    one = map_chr(., c("x", "one")),
    three = map_chr(., c("y", "two", "three", "four", "five"), .null = NA_character_)
)

Works as expected.

edited Nov 22 '19 at 13:46

Wai Ha Lee

8,598
83
57
92

answered Nov 22 '19 at 12:33

Martin Andersen

1
2

How do I iterate over a nested list, where nodes may be missing

1 Answers1