How to import and save multiple data files with a for loop in R?

Question

I'm trying to import data for multiple rounds using the package "openair". This is the code I'm using

install.packages("devtools")
devtools::install_github("r-lib/conflicted")


#openair
library(openair)
library(tidyverse)

#import
sites_2005_2019 <- importMeta(
  source = "aurn",
  year = 2005:2019
)

 defra_aurn[[y]] <- for (y in 2005:2019) {
  
importAURN(year = y,
                         data_type = "daily",
                         site=sites_2005_2019$code,
                         meta = TRUE,

 save(defra_aurn[[y]], file = "C:/Users/..../defra_aurn.Rdata")                        
)
}

this gives the error:

Error in save(defra_aurn[[y]], file = "C:/Users/..../defra_aurn[[y]].Rdata") : 
  object ‘defra_aurn[[y]]’ not found

I've tried putting the save command outside the loop, inside the loop but outside the argument,but I nevertheless get error that object not found.

I cannot simply use 2005:2019 in the argument to import as that would only import sites that were open during the whole period, but I need all available sites in each year, so I think the loop is the only way to go forward.

Would appreciate any help, thanks!

I described the problem in the previous section. Hope that helps, I'd be happy to add more details if needed

adding more details here:

thanks this worked, but unable to save it in r and then write it to stata.

this code is giving error in the last line: 

Error in write.dta(defra_aurn, "defra_aurn.dta") : 
  The object "dataframe" must have class data.frame

To correct this I tried numerous options like

defra_aurn_combined <- do.call(rbind, defra_aurn)

defra_aurn_combined <- bind_rows(defra_aurn)

full_join()

and several variants of these.

I have added some details to the original post One challenge of combining these data frames is that number of sites vary over dates, so either I get errors that numbers of columns of arguments do not match, or the combined data has only sites common to all years, which is not what I'm looking for.

Taren Sanders · Answer 1 · 2023-07-09T23:34:55.687

0

There's a couple of problems here. The save function can't access the object because it doesn't exist yet, and you cannot assign using y because y only exists in the loop.

What you want is lapply and to save afterwards. Here's a reduced example:

# openair
library(openair)
library(tidyverse)

# import
sites_2005_2019 <- importMeta(
  source = "aurn",
  year = 2005:2019
)

sites_2005_2019 <- head(sites_2005_2019, 3)

defra_aurn <- lapply(setNames(c(2005:2006), c(2005:2006)), function(y) {
  importAURN(
    year = y,
    data_type = "daily",
    site = sites_2005_2019$code,
    meta = TRUE
  )
})

save(defra_aurn, file = "defra_aurn.Rdata")

Note that I used setNames so that you get back a named list. Also, you only need to save once because you are saving the whole object (the named list), rather than each item in the list.

Edit: In the comments you say you want to then save this to read into stata. To do this you need a data.frame not a named list. One option would be to add this to the end:

full_dataset <- do.call(rbind, defra_aurn)
# full_dataset <- purrr::list_rbind(defra_aurn) # Or purrr alternative
foreign::write.dta(full_dataset, "full_dataset.dta")
save(defra_aurn, file = "defra_aurn.Rdata")

edited Jul 09 '23 at 23:34

answered Jul 06 '23 at 01:35

Taren Sanders

421
2
12

thanks this worked, but unable to save it in r and then write it to stata. ``` this code is giving error in the last line: Error in write.dta(defra_aurn, "defra_aurn.dta") : The object "dataframe" must have class data.frame ``` To correct this I tried numerous options like ``` defra_aurn_combined <- do.call(rbind, defra_aurn) ``` ``` – Jan3 Jul 06 '23 at 15:00
@Jan3 that's a different issue. The object "defra_aurn" is a named list and write.dta is expecting a data.frame. If the goal was to end up with a data.frame to write to Stata you should have included it in the question - there's easier ways to accomplish it. I'll edit my answer. – Taren Sanders Jul 06 '23 at 22:53
@Jan3 if that answers your question please mark this as the solution. – Taren Sanders Jul 06 '23 at 22:55
thanks for the solution. this is giving an error at the last line : ``` Error in rbind(deparse.level, ...) : numbers of columns of arguments do not match ``` – Jan3 Jul 07 '23 at 17:32
If your dataframes are not always the same you could use purrr::list_rbind which is more robust than base rbind (https://purrr.tidyverse.org/reference/list_c.html) – Taren Sanders Jul 09 '23 at 23:35

How to import and save multiple data files with a for loop in R?

1 Answers1