Can I use group_map or group_walk to iteratively export results?

Question

I want to iteratively process a master list of comparisons using group_walk() or group_map() as an alternative method to import batches of .csv files.

I would like to input a dataset that looks like this:

Test Assay	Var1	Var2	Freq
Assay1	neg	neg	19
Assay1	neg	pos	5
Assay1	pos	neg	8
Assay1	pos	pos	141
Assay2	neg	neg	25
Assay2	neg	pos	6
Assay2	pos	neg	17
Assay2	pos	pos	33
Assay3	neg	neg	99
Assay3	neg	pos	20
Assay3	pos	neg	5
Assay3	pos	pos	105

I want to use the function epi_analysis and export a csv for each Test Assay (in this example Assay1, Assay2, and Assay3). So far I have:

#Make export directory
check_create_dir <- function(the_dir) {
  if (!dir.exists(the_dir)) {
    dir.create(the_dir, recursive = TRUE) } #Creates a directory if it doesn't already exist
}

the_dir_ex <- "data_generated/epidata" #Name the new desired directory

check_create_dir(the_dir_ex) #Make the directory if it doesn't already exist

#Make function for the series of analyses
epi_analysis <- function(.x, the_dir){
  #Clean data
  dat2 <- .x  %>%
    select(c(Var1, Var2, Freq)) %>%
    pivot_wider(Var1, names_from = Var2, values_from = Freq) %>%
    remove_rownames %>% 
    column_to_rownames( var = "Var1") %>% 
    as.matrix() 
  
  #Run tests
  rval <- epi.tests(dat2, conf.level = 0.95)
  rkappa<-epi.kappa(dat2)
  gwet <- gwet.ac1.table(dat2)
  kappa2 <- kappa2.table(dat2)
  
  #Export results
  hd <- c('sensitivity', 'specificity', 'pfp', 'pfn', 'kappa', 'gwet', 'pabak')
  ests <- c(round(rval$elements$sensitivity$est, digits = 3), 
            round(rval$elements$specificity$est, digits = 3), 
            round(rval$element$pfp$est, digits = 3), 
            round(rval$element$pfn$est, digits = 3), 
            round(kappa2$coeff.val, digits = 3), 
            round(gwet$coeff.val, digits = 3), 
            round(rkappa$pabak$est, digits = 3))
  cis <- c(paste(round(rval$elements$sensitivity$lower, digits = 3), round(rval$elements$sensitivity$upper, digits = 3), sep = ","), 
           paste(round(rval$elements$specificity$lower, digits = 3), round(rval$elements$specificity$upper, digits = 3), sep = ","),
           paste(round(rval$element$pfp$lower, digits = 3), round(rval$element$pfp$upper, digits = 3), sep = ","),  
           paste(round(rval$element$pfn$lower, digits = 3), round(rval$element$pfn$upper, digits = 3), sep = ","), 
           kappa2$coeff.ci, 
           gwet$coeff.ci, 
           paste(round(rkappa$pabak$lower, digits = 3), round(rkappa$pabak$lower, digits = 3), sep = ","))
  
  df <- data.frame(hd, ests, cis)
  
  write.csv(df, 
            file = paste0(the_dir, "/", basename(.x$TestAssay)),
            na = "999.99", 
            row.names = FALSE)
  
}


#Use group_map or group_walk to iterate through the different assays in the dataset.

data <- read_csv("data_raw/EpiTest.csv") %>%
  group_by(TestAssay)%>%
  group_map(~ epi_analysis)

But there are no csvs in my epidata folder. Any suggestions/corrections welcomed.

score 2 · Answer 1 · answered Jun 05 '21 at 18:19

2

We can use

library(dplyr)
library(readr)
library(purrr)
read_csv("data_raw/EpiTest.csv") %>%
   group_split(TestAssay) %>%
   map(~ epi_analysis(.x, the_dir_ex))

answered Jun 05 '21 at 18:19

akrun

874,273
37
540
662

Ronak Shah · Accepted Answer · 2021-06-06T01:50:38.783

You need to call your function in group_map. Also the function requires two arguments so pass the_dir_ex as well.

Use this function -

library(tidyverse)
library(epiR)
library(irrCAC)


epi_analysis <- function(.x, the_dir){
dat2 <- .x  %>%
  select(c(Var1, Var2, Freq)) %>%
  pivot_wider(Var1, names_from = Var2, values_from = Freq) %>%
  remove_rownames %>% 
  column_to_rownames( var = "Var1") %>% 
  as.matrix() 

#Run tests
rval <- epi.tests(dat2, conf.level = 0.95)
rkappa<-epi.kappa(dat2)
gwet <- gwet.ac1.table(dat2)
kappa2 <- kappa2.table(dat2)

#Export results
hd <- c('sensitivity', 'specificity', 'pfp', 'pfn', 'kappa', 'gwet', 'pabak')
ests <- c(round(rval$elements$sensitivity$est, digits = 3), 
          round(rval$elements$specificity$est, digits = 3), 
          round(rval$element$pfp$est, digits = 3), 
          round(rval$element$pfn$est, digits = 3), 
          round(kappa2$coeff.val, digits = 3), 
          round(gwet$coeff.val, digits = 3), 
          round(rkappa$pabak$est, digits = 3))
cis <- c(paste(round(rval$elements$sensitivity$lower, digits = 3), round(rval$elements$sensitivity$upper, digits = 3), sep = ","), 
         paste(round(rval$elements$specificity$lower, digits = 3), round(rval$elements$specificity$upper, digits = 3), sep = ","),
         paste(round(rval$element$pfp$lower, digits = 3), round(rval$element$pfp$upper, digits = 3), sep = ","),  
         paste(round(rval$element$pfn$lower, digits = 3), round(rval$element$pfn$upper, digits = 3), sep = ","), 
         kappa2$coeff.ci, 
         gwet$coeff.ci, 
         paste(round(rkappa$pabak$lower, digits = 3), round(rkappa$pabak$lower, digits = 3), sep = ","))

df <- data.frame(hd, ests, cis)

write.csv(df, 
          file = sprintf('%s/%s.csv', the_dir, .x$TestAssay[1]),
          na = "999.99", 
          row.names = FALSE)

}

and call it with -

read_csv("data_raw/EpiTest.csv") %>%
  group_by(TestAssay)%>%
  group_map(~epi_analysis(., the_dir_ex), .keep = TRUE)

That makes sense, but when I ran it I received this error Error in basename(.x$TestAssay) : a character vector argument expected In addition: Warning message: Error in basename(.x$TestAssay) : a character vector argument expected. Do I need to fix my function? — MBell, Jun 05 '21 at 13:43
I think so. Does your function work without `group_by` ? Try with `epi_analysis(df, the_dir_ex)` — Ronak Shah, Jun 05 '21 at 14:13
So no, If I run `data <- read_csv("data_raw/EpiTest_subset") epi_analysis(data, the_dir_ex)`, I get `Error in UseMethod("select_") : no applicable method for 'select_' applied to an object of class "function"` . I think the problem lies in the clean data part of the function. I have to take each "Assay" and then make them into individual contingency table matrices. I can do this using that pipe for a single dataset, but it doesn't seem to be working well within the function — MBell, Jun 05 '21 at 16:47
@MBell There were few changes required in the function as well as in `group_map`. I have corrected that and edited the answer. Can you try the updated answer? I tested the answer for the data you have shared and got 3 csvs as output. — Ronak Shah, Jun 06 '21 at 01:51
That worked perfectly, thank you! Out of curiosity what was the difference between paste0 and sprintf in the function? — MBell, Jun 09 '21 at 14:31
There is no difference. It's just that I find using `sprintf` easier. — Ronak Shah, Jun 09 '21 at 14:51

Can I use group_map or group_walk to iteratively export results?

2 Answers2

Linked