Dplyr function to compute average, n, sd and standard error

Question

I find myself writing this bit of code all the time to produce standard errors for group means ( to then use for plotting confidence intervals).

It would be nice to write my own function to do this in one line of code, though. I have read the nse vignette in dplyr on non-standard evaluation and this blog post as well. I get it somewhat, but I'm too much of a noob to figure this out on my own. Can anyone help out? Thanks.

var1<-sample(c('red', 'green'), size=10, replace=T)
var2<-rnorm(10, mean=5, sd=1)
df<-data.frame(var1, var2)
df %>% 
group_by(var1) %>% 
summarize(avg=mean(var2), n=n(), sd=sd(var2), se=sd/sqrt(n))

Can you show what you have tried? Where did you get stuck? Have a look at some of the questions in [nse] tag. — Axeman, May 30 '17 at 15:46
Well, I was playing around with this code in the blog post: `code`mean_mpg = function(data, ..., x) { data %>% group_by_(.dots = lazyeval::lazy_dots(...)) %>% summarize(mean_mpg = ~mean(x)) } mtcars %>% mean_mpg(cyl, gear, mpg) `code` It returned the error Not a Vector — spindoctor, May 30 '17 at 16:23

tbradley · Accepted Answer · 2017-10-30T15:50:33.750

You can use the function enquo to explicitly name the variables in your function call:

my_fun <- function(x, cat_var, num_var){
  cat_var <- enquo(cat_var)
  num_var <- enquo(num_var)

  x %>%
    group_by(!!cat_var) %>%
    summarize(avg = mean(!!num_var), n = n(), 
              sd = sd(!!num_var), se = sd/sqrt(n))
}

which gives you:

> my_fun(df, var1, var2)
# A tibble: 2 x 5
    var1      avg     n        sd        se
  <fctr>    <dbl> <int>     <dbl>     <dbl>
1  green 4.873617     7 0.7515280 0.2840509
2    red 5.337151     3 0.1383129 0.0798550

and that matches the ouput of your example:

> df %>% 
+   group_by(var1) %>% 
+   summarize(avg=mean(var2), n=n(), sd=sd(var2), se=sd/sqrt(n))
# A tibble: 2 x 5
    var1      avg     n        sd        se
  <fctr>    <dbl> <int>     <dbl>     <dbl>
1  green 4.873617     7 0.7515280 0.2840509
2    red 5.337151     3 0.1383129 0.0798550

EDIT:

The OP has asked to remove the group_by statement from the function to add the ability to group_by more than one variables. There are two ways to go about this IMO. First, you could simply remove the group_by statement and pipe a grouped data frame into the function. That method would look like this:

my_fun <- function(x, num_var){
  num_var <- enquo(num_var)

  x %>%
    summarize(avg = mean(!!num_var), n = n(), 
              sd = sd(!!num_var), se = sd/sqrt(n))
}

df %>%
  group_by(var1) %>%
  my_fun(var2)

Another way to go about this is to use ... and quos to allow for the function to capture multiple arguments for the group_by statement. That would look like this:

#first, build the new dataframe
var1<-sample(c('red', 'green'), size=10, replace=T)
var2<-rnorm(10, mean=5, sd=1)
var3 <- sample(c("A", "B"), size = 10, replace = TRUE)
df<-data.frame(var1, var2, var3)

# using the first version `my_fun`, it would look like this
df %>%
  group_by(var1, var3) %>%
  my_fun(var2)

# A tibble: 4 x 6
# Groups:   var1 [?]
    var1   var3      avg     n        sd        se
  <fctr> <fctr>    <dbl> <int>     <dbl>     <dbl>
1  green      A 5.248095     1       NaN       NaN
2  green      B 5.589881     2 0.7252621 0.5128378
3    red      A 5.364265     2 0.5748759 0.4064986
4    red      B 4.908226     5 1.1437186 0.5114865

# Now doing it with a new function `my_fun2`
my_fun2 <- function(x, num_var, ...){
  group_var <- quos(...)
  num_var <- enquo(num_var)

  x %>%
    group_by(!!!group_var) %>%
    summarize(avg = mean(!!num_var), n = n(), 
              sd = sd(!!num_var), se = sd/sqrt(n))
}

df %>%
  my_fun2(var2, var1, var3)

# A tibble: 4 x 6
# Groups:   var1 [?]
    var1   var3      avg     n        sd        se
  <fctr> <fctr>    <dbl> <int>     <dbl>     <dbl>
1  green      A 5.248095     1       NaN       NaN
2  green      B 5.589881     2 0.7252621 0.5128378
3    red      A 5.364265     2 0.5748759 0.4064986
4    red      B 4.908226     5 1.1437186 0.5114865

You should probably note that this only works in the dev version of `dplyr`, _not_ the current CRAN version, which OP is most likely using. — Axeman, May 30 '17 at 21:21
I'm finally returning to this; I had forgotten I had asked this. But is it possible to *not* include the categorical grouping variables in the function? Sometimes I group by one, sometimes by two grouping variables. I'd like to keep that flexibility outside the custom function. But I don't know if that is possible. — spindoctor, Oct 30 '17 at 15:09
I have added an edit that will let you do this in 2 different ways — tbradley, Oct 30 '17 at 15:50
This is great, an I have been using this but I feel like a function like this should be in some package somwhere. Does anyone know if this is somewhere in a tidyverse friendly package? — spindoctor, Sep 23 '22 at 14:22

Dplyr function to compute average, n, sd and standard error

1 Answers1

Linked