Populating a data frame in R in a loop

Question

I am trying to populate a data frame from within a for loop in R. The names of the columns are generated dynamically within the loop and the value of some of the loop variables is used as the values while populating the data frame. For instance the name of the current column could be some variable name as a string in the loop, and the column can take the value of the current iterator as its value in the data frame.

I tried to create an empty data frame outside the loop, like this

d = data.frame()

But I cant really do anything with it, the moment I try to populate it, I run into an error

 d[1] = c(1,2)
Error in `[<-.data.frame`(`*tmp*`, 1, value = c(1, 2)) : 
  replacement has 2 rows, data has 0

What may be a good way to achieve what I am looking to do. Please let me know if I wasnt clear.

Answer 1

It is often preferable to avoid loops and use vectorized functions. If that is not possible there are two approaches:

1. Preallocate your data.frame. This is not recommended because indexing is slow for data.frames.
2. Use another data structure in the loop and transform into a data.frame afterwards. A list is very useful here.

Example to illustrate the general approach:

mylist <- list() #create an empty list

for (i in 1:5) {
  vec <- numeric(5) #preallocate a numeric vector
  for (j in 1:5) { #fill the vector
    vec[j] <- i^j 
  }
  mylist[[i]] <- vec #put all vectors in the list
}
df <- do.call("rbind",mylist) #combine all vectors into a matrix

In this example it is not necessary to use a list, you could preallocate a matrix. However, if you do not know how many iterations your loop will need, you should use a list.

Finally here is a vectorized alternative to the example loop:

outer(1:5,1:5,function(i,j) i^j)

As you see it's simpler and also more efficient.

Answer 2

You could do it like this:

 iterations = 10
 variables = 2

 output <- matrix(ncol=variables, nrow=iterations)

 for(i in 1:iterations){
  output[i,] <- runif(2)

 }

 output

and then turn it into a data.frame

 output <- data.frame(output)
 class(output)

what this does:

1. create a matrix with rows and columns according to the expected growth
2. insert 2 random numbers into the matrix
3. convert this into a dataframe after the loop has finished.

Answer 3

this works too.

df = NULL
for (k in 1:10)
    {
       x = 1
       y = 2
       z = 3
       df = rbind(df, data.frame(x,y,z))
     }

output will look like this

df #enter

x y z #col names
1 2 3

Answer 4

Thanks Notable1, works for me with the tidytextr Create a dataframe with the name of files in one column and content in other.

    diretorio <- "D:/base"
    arquivos <- list.files(diretorio, pattern = "*.PDF")
    quantidade <- length(arquivos)

#
df = NULL
for (k in 1:quantidade) {

      nome = arquivos[k]
      print(nome)
      Sys.sleep(1) 
      dados = read_pdf(arquivos[k],ocr = T)
      print(dados)
      Sys.sleep(1)
      df = rbind(df, data.frame(nome,dados))
      Sys.sleep(1)
}
Encoding(df$text) <- "UTF-8"

Answer 5

I had a case in where I was needing to use a data frame within a for loop function. In this case, it was the "efficient", however, keep in mind that the database was small and the iterations in the loop were very simple. But maybe the code could be useful for some one with similar conditions.

The for loop purpose was to use the raster extract function along five locations (i.e. 5 Tokio, New York, Sau Paulo, Seul & Mexico city) and each location had their respective raster grids. I had a spatial point database with more than 1000 observations allocated within the 5 different locations and I was needing to extract information from 10 different raster grids (two grids per location). Also, for the subsequent analysis, I was not only needing the raster values but also the unique ID for each observations.

After preparing the spatial data, which included the following tasks:

1. Import points shapefile with the readOGR function (rgdap package)
2. Import raster files with the raster function (raster package)
3. Stack grids from the same location into one file, with the function stack (raster package)

Here the for loop code with the use of a data frame:

1. Add stacked rasters per location into a list

raslist <- list(LOC1,LOC2,LOC3,LOC4,LOC5)

2. Create an empty dataframe, this will be the output file

TB <- data.frame(VAR1=double(),VAR2=double(),ID=character())

3. Set up for loop function

L1 <- seq(1,5,1) # the location ID is a numeric variable with values from 1 to 5 

for (i in 1:length(L1)) {
  dat=subset(points,LOCATION==i) # select corresponding points for location [i] 
  t=data.frame(extract(raslist[[i]],dat),dat$ID) # run extract function with points & raster stack for location [i]
  names(t)=c("VAR1","VAR2","ID") 
  TB=rbind(TB,t)
}

Answer 6

was looking for the same and the following may be useful as well.

a <- vector("list", 1)
for(i in 1:3){a[[i]] <- data.frame(x= rnorm(2), y= runif(2))}
a
rbind(a[[1]], a[[2]], a[[3]])