Calculating slope and associated stats by rows using multiple categories in dataframe that has NAs

Question

I have looked at related questions posted under: "How can I calculate the slope of multiple subsets of a data frame more efficiently?" and my beginner status doesn't allow me to comment on that thread directly (not sure how to do that), so I ask here:

How do you avoid NAs in dataset to process calculation of slopes across multiple categories in a dataframe using the dplyr&broom package solutions? Here is a sample of script and results?

SAMPLE DATA:

DOY<-c(102,102,102,102,102,102,102,102,102,102,212,212,212,212,212,212, 212,212,212,212)
LOCATION <- c(1,1,1,1,1,2,2,2,2,2,1,1,1,1,1,3,3,3,3,3)
response <-c(NA,NA,NA,NA,NA,7,10,15,20,30,2,4,6,NA,8,10,15,20,30,NA) 
ts <- c(0,10,20,30, 40,0,10,20,30,40,0,10,20,30,40,0,10,20,30,40)
test.data <- data.frame(cbind(DOY, LOCATION, response, ts))

    library(dplyr)
    library(broom) 

 test.data2 <- test.data %>%  group_by(DOY) %>% do(tidy(lm(response ~ ts, data = .))) 
   test.data2 %>% filter(term == "ts")

RESULT FOR ONE CONDITION WORKS (as there is enough data per row without NAs):

# A tibble: 2 x 6
   # Groups:   DOY [2]
   #            DOY    term  estimate    std.error   statistic   p.value
   #            <dbl>  <chr>    <dbl>     <dbl>       <dbl>       <dbl>
   #     1     102.     ts       0.560    0.0721      7.77      0.00444
   #     2     212.     ts       0.278    0.247       1.13      0.303

BUT IF MULTIPLE CATEGORIES ARE USED TO GROUP, THEN NOT:

test.dataX <- test.data %>%  group_by(LOCATION, DOY) %>% do(tidy(lm(response ~ ts, data = .)))

RESULTS in Errors:

   # Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : 
   #  0 (non-NA) cases

 test.dataX %>% filter(term == "ts")
   # Error in eval(lhs, parent, parent) : object 'test.dataX' not found

ATTEMPT 2: I tried na.omit in lm(), but that also didn't work:

test.dataX <- test.data %>%  group_by(LOCATION, DOY) %>% do(tidy(lm(response ~ ts, data = ., na.action=na.omit)))
   # Error in lm.fit(x, y, offset = offset, singular.ok = singular.ok, ...) : 
   #  0 (non-NA) cases

IDEALLY I WOULD LIKE TO HAVE SOMETHING LIKE THIS (together with R2 if possible - how do I add that to the output above)?

# DOY   LOCATION    slope     R2
 # 102    1         NA        NA
 # 102    2         0.560     0.953
 # 212    1         0.149     0.966
 # 212    3         0.650     0.966

########################

PLEASE ADVISE. THANK YOU!

akrun · Accepted Answer · 2018-06-09T04:22:18.337

If we want to return NA, then use possibly

library(tidyverse)
library(broom)
pos1 <-  possibly(lm, otherwise = NULL)
prsq <- possibly(pull, otherwise = NA)
test.data %>%
     group_by(DOY, LOCATION) %>%
     nest %>%
     mutate(model = map(data, ~ pos1(response~ ts, data = .x)),
            slope = map_dbl(model, ~ 
                            .x %>% 
                                tidy %>%
                                filter(term == 'ts') %>%
                                prsq(estimate)),
            R2 = map_dbl(model, ~ 
                             .x %>%
                                   glance %>%
                                   prsq(r.squared))) %>%
      select(-data, -model)
# A tibble: 4 x 4
#    DOY LOCATION  slope     R2
#  <dbl>    <dbl>  <dbl>  <dbl>
#1   102        1 NA     NA    
#2   102        2  0.56   0.953
#3   212        1  0.149  0.966
#4   212        3  0.650  0.966

Calculating slope and associated stats by rows using multiple categories in dataframe that has NAs

1 Answers1