Tidyverse solution for Counting nulls from multiple fields

Question

I'm trying to summarize my data by counting the number of nulls and number of responses (values + NA) for 12 different columns. (renamed for this example)

Data:

my_df <- structure(list(SCR1 = c(100, 80.2, 70.3, 
90.1, 100, 100, 100, 100, 40.6, 80.2, 80.2, 90.1, 30.7, 100, 
80.2), SCR2 = c(75.25, 75.25, 75.25, 75.25, 75.25, 75.25, 
100, 100, 50.5, 50.5, 75.25, 50.5, 50.5, 100, 50.5), SCR3 = c(100, 
100, 100, 100, 75.25, 75.25, 100, 100, 100, 75.25, 100, 100, 
NA, 100, 75.25), SCR4 = c(100, 50.5, 
50.5, 75.25, 100, 75.25, 100, 100, 100, 75.25, 50.5, 100, 100, 
100, 75.25), SCR5 = c(100, 75.25, 50.5, 
75.25, 100, 75.25, 100, 100, 50.5, 75.25, 75.25, 75.25, 25.75, 
100, 75.25), SCR6 = c(100, 25.75, 50.5, 
75.25, 100, 75.25, 100, 100, 75.25, 75.25, 75.25, 100, 50.5, 
100, 50.5), SCR7 = c(75.25, 50.5, 75.25, 
100, 50.5, 75.25, 100, 100, 75.25, 50.5, 75.25, 100, 25.75, 100, 
75.25), SCR8 = c(75.25, 75.25, 50.5, 75.25, 
75.25, 75.25, 100, 100, 75.25, 75.25, 75.25, 75.25, 50.5, 100, 
75.25), SCR9 = c(50.5, NA, NA, 25.75, 100, NA, 
NA, NA, 100, NA, NA, NA, 75.25, NA, NA), SCR10 = c(25.75, 
NA, NA, NA, NA, 1, NA, NA, NA, NA, NA, NA, NA, NA, NA), SCR11 = c(100, 
75.25, 50.5, 75.25, 100, 75.25, 100, 100, 75.25, 75.25, 75.25, 
100, 50.5, 100, 50.5), SCR12 = c(75.25, 75.25, 
50.5, NA, 75.25, 75.25, 100, 100, 75.25, NA, 75.25, 75.25, 50.5, 
100, 50.5), ID = 1:15), row.names = c(NA, 15L), class = "data.frame")

Here I'd like to return the number of values and the number of NA and then the percent NA

I found a solution to count the number of NA but I'm struggling to add a 2nd/3rd column with count and percents.

my_df %>% map_df(function(x) sum(is.na(x))) %>% gather(feature, num_nulls) %>% dplyr::arrange(desc(num_nulls))

How can I add counts and percent na? I'm trying to use the tidyverse and avoid using SQL.

`responses` = the number of values including NA. For this example each column has 15 `responses` — Ryan John, Aug 28 '18 at 14:51

score 3 · Accepted Answer · answered Aug 28 '18 at 14:52

You are almost there. The additional two columns could be created with an additional mutate statement:

my_df %>%
    map_df(function(x) sum(is.na(x))) %>%
    gather(feature, num_nulls) %>%
    dplyr::arrange(desc(num_nulls)) %>%
    mutate(num_responses = nrow(my_df) - num_nulls,
        percent = num_responses/nrow(my_df))

Tidyverse solution for Counting nulls from multiple fields

1 Answers1

Linked