How do I show individual points of a boxplot in R?

Question

I have the df1:

              Name        Y_N FIPS  score1 score2
 1:        Alabama         0    1   2633      8
 2:         Alaska         0    2    382      1
 3:        Arizona         1    4   2695     41
 4:       Arkansas         1    5   2039     10
 5:     California         1    6  27813    524
 6:       Colorado         0    8   8609    133
 7:    Connecticut         1    9   5390    111
 8:       Delaware         0   10    858      3
 9:        Florida         1   12  14172    215
10:        Georgia         1   13   9847    308
11:         Hawaii         0   15    720      0
12:          Idaho         1   16    845      7

I would like to perform a T-test to see if score1 differs based on Y_N. I would then like to plot these two against each other. I have made a boxplot that looks like:

Instead I want my graph to look like except with confidence bars: I want to now change from a boxplot to a plot that shows all of the individual points and then a mean horizontal line with 95% confidence intervals. How is this done? I would also like to add the text of the p-value in a corner of the graph.

I might try:

text(x = max(df1$Y_N)+1, 
     y = min(df1$score1)+20000, 
     labels = paste0(
                     "\np-value = ",
                     round(coef_lm[2,4],5),            
     pos = 4)

But I realize that coef_lm[2,4],5 are the test-statistics from a linear model. How do I access the outputs of a t-test?

Now that you've provided your very nice paint vison for the figure, please see my edit. — Ian Campbell, Mar 30 '20 at 16:23

Ian Campbell · Accepted Answer · 2020-03-30T16:17:26.057

I'm not sure why you added that extra point in your code. But on your original data, you might use ggplot2 and ggpubr.

Edit Now more like your paint drawing.

ggplot(df1,aes(x = as.factor(Y_N), y = score1)) + 
  geom_jitter(position = position_jitter(0.1)) + 
  stat_summary(fun.data = "mean_cl_normal", geom = "errorbar", width = 0.3) +
  stat_summary(fun = "mean", geom = "errorbar",  aes(ymax = ..y.., ymin = ..y..), col = "red", width = 0.5) +
  stat_compare_means(method="t.test") + 
  xlab("Group") + ylab("Score 1")

Original Data

df1 <- structure(list(Name = structure(1:12, .Label = c("Alabama", "Alaska", 
"Arizona", "Arkansas", "California", "Colorado", "Connecticut", 
"Delaware", "Florida", "Georgia", "Hawaii", "Idaho"), class = "factor"), 
    Y_N = c(0L, 0L, 1L, 1L, 1L, 0L, 1L, 0L, 1L, 1L, 0L, 1L), 
    FIPS = c(1L, 2L, 4L, 5L, 6L, 8L, 9L, 10L, 12L, 13L, 15L, 
    16L), score1 = c(2633L, 382L, 2695L, 2039L, 27813L, 8609L, 
    5390L, 858L, 14172L, 9847L, 720L, 845L), score2 = c(8L, 1L, 
    41L, 10L, 524L, 133L, 111L, 3L, 215L, 308L, 0L, 7L)), class = "data.frame", row.names = c("1:", 
"2:", "3:", "4:", "5:", "6:", "7:", "8:", "9:", "10:", "11:", 
"12:"))

Thanks Ian! I was wondering if I wanted to t-test not based on the Y_N column, but instead based on if the `score2` column is greater than 100, how could I do that? — Evan, Apr 01 '20 at 17:07

dc37 · Answer 2 · 2020-03-30T16:05:02.397

Alternatively, without to install ggpubr you can calculate p value outside of ggplot2 and use annotate function to add the pvalue into the plot:

pval <- t.test(score1~Y_N,data = df)$p.value

library(ggplot2)
ggplot(df, aes(x = as.factor(Y_N), y = score1, fill = as.factor(Y_N), color = as.factor(Y_N)))+
  geom_boxplot(alpha = 0.3, color = "black", outlier.shape = NA)+
  geom_jitter(show.legend = FALSE)+
  annotate(geom = "text", label = paste("p.value: ",round(pval,3)), x = 1.5, y = max(df$score1)*0.9)

EDIT: Without a boxplot

Alternatively to the boxplot, if you want to have individual points and a bar representing the mean, you can first calculate the mean per group in a ne dataset (here I'm using dplyr package for doing it):

library(dplyr)
Mean_df <- df %>% group_by(Y_N) %>% summarise(Mean = mean(score1))

# A tibble: 2 x 2
    Y_N  Mean
  <int> <dbl>
1     0 2640.
2     1 8972.

Then, you can plot individual points using geom_jitter and the mean using geom_errobar by calling the new dataset Mean_df:

library(ggplot2)
ggplot(df, aes(x = as.factor(Y_N), y = score1))+
  geom_jitter(show.legend = FALSE, width = 0.2)+
  geom_errorbar(inherit.aes = FALSE, data = Mean_df, 
                aes(x = as.factor(Y_N),ymin = Mean, ymax = Mean),
                color = "red",width = 0.2)+
  annotate(geom = "text", label = paste("p.value: ",round(pval,3)), 
           x = 1.5, y = max(df$score1)*0.9)

Reproducible example

structure(list(Name = c("Alabama", "Alaska", "Arizona", "Arkansas", 
"California", "Colorado", "Connecticut", "Delaware", "Florida", 
"Georgia", "Hawaii", "Idaho"), Y_N = c(0L, 0L, 1L, 1L, 1L, 0L, 
1L, 0L, 1L, 1L, 0L, 1L), FIPS = c(1L, 2L, 4L, 5L, 6L, 8L, 9L, 
10L, 12L, 13L, 15L, 16L), score1 = c(2633L, 382L, 2695L, 2039L, 
27813L, 8609L, 5390L, 858L, 14172L, 9847L, 720L, 845L), score2 = c(8L, 
1L, 41L, 10L, 524L, 133L, 111L, 3L, 215L, 308L, 0L, 7L)), row.names = c(NA, 
-12L), class = c("data.table", "data.frame"))

@Evan, I edited my answer. Let me know if it is what you are looking for — dc37, Mar 30 '20 at 16:07
How do I change the x-axis label from saying "as.factor", and how do I get the error bars? — Evan, Mar 30 '20 at 16:16
You should take a look at @IanCampbell's answer. it answer all your questions. You should validate his/her answer. — dc37, Mar 30 '20 at 16:23

score 1 · Answer 3 · answered Mar 30 '20 at 16:21

dd <- structure(list(Name = c("Alabama", "Alaska", "Arizona", "Arkansas",  "California", "Colorado", "Connecticut", "Delaware", "Florida",  "Georgia", "Hawaii", "Idaho"), Y_N = c(0L, 0L, 1L, 1L, 1L, 0L,  1L, 0L, 1L, 1L, 0L, 1L), FIPS = c(1L, 2L, 4L, 5L, 6L, 8L, 9L,  10L, 12L, 13L, 15L, 16L), score1 = c(2633L, 382L, 2695L, 2039L,  27813L, 8609L, 5390L, 858L, 14172L, 9847L, 720L, 845L), score2 = c(8L,  1L, 41L, 10L, 524L, 133L, 111L, 3L, 215L, 308L, 0L, 7L)), row.names = c(NA,  -12L), class = c("data.table", "data.frame"))

## frame
boxplot(score1 ~ Y_N, dd, border = NA)

## 95% ci, medians
sp <- split(dd$score1, dd$Y_N)
sapply(seq_along(sp), function(ii) {
  x <- sp[[ii]]
  arrows(ii, quantile(x, 0.025), ii, quantile(x, 0.975), code = 3, angle = 90, length = 0.1)
  segments(ii - 0.05, median(x), ii + 0.05, col = 'red', lwd = 2)
})

points(dd$Y_N + 1, dd$score1, col = dd$Y_N + 1)

## t-test
lbl <- sprintf('p = %s', format.pval(t.test(score1 ~ Y_N, dd)$p.value, digits = 2))
mtext(lbl, at = par('usr')[2], adj = 1)

Chris Ruehlemann · Answer 4 · 2020-03-30T16:13:37.550

One of your questions relates to how to access the t.test statistics. Here's an answer to that question. Suppose you have that type of data:

set.seed(12)
YN <- sample(0:1, 100, replace = T)    
score1 <- sample(500:1500, 100, replace = T)
df <- data.frame(YN, score1)

And suppose further that you run and store the t.test like this:

test <- tapply(df$score1, df$YN, t.test)

Then you can access the test statistics bit by bit like this, illustrated here for the factor level 0:

test$`0`$p.value #   p-value
test$`0`$conf.int #  confidence interval
test$`0`$estimate #  estimate
test$`0`$statistic # statistic

Now obviously you will not want to do it manually bit by bit but in a more autmated and systematic way. This is how you can achieve this:

df1 <- do.call(rbind, lapply(test, function(x) c(
  statistic = unname(x$statistic),
  ci = unname(x$conf.int),
  est = unname(x$estimate),
  pval = unname(x$p.value))))

The ouput is this:

  statistic      ci1      ci2      est         pval
0  22.31155 837.3901 1003.263 920.3265 5.484012e-27
1  22.91558 870.5426 1037.810 954.1765 3.543693e-28

Interesting use of `tapply`. However, why do you have two p.values ? And how do you get the p.value of the difference between two groups (which is what the OP is looking for I guess) ? Does `t.test(....)$p.value` not simpler? — dc37, Mar 30 '20 at 16:13

How do I show individual points of a boxplot in R?

4 Answers4