1

I have a huge set of data that I have distributed on a list (33 years of hourly precipitation data for South America). Every object of the list is an ff array of data (one array per year; I have to use the ff package to avoid running out of RAM). Then, I need to select certain hours of every year, that is, certain matrices of every array, and paste them together. With a normal list I would do this using lapply:

require (abind)

pp_hour <- list()
length(pp_hour) <- 24

# pp  is the list that contains the data.
# pp[[i]] is an array of hourly data for the i year. It has 3 dimensions: longitude, latitude and time. 

for (j in 0:23) { # with this loop I intend to select all the j-hours and paste them together
  t_aux <- c(0:23) == j
  pp_hour[[j+1]] <- abind( lapply( pp, "[", ,,t_aux))
 }

This method doesn't work with the ff objects, I get the error: 

Error in eval(expr, envir, enclos) : argument is missing, with no default

I see there is an ff variant of apply, ffapply, but no variant of lapply. Anyone knows any elegant workaround?

JulianGiles
  • 328
  • 2
  • 7
  • 1
    There's no way to pass empty arguments through `lapply` AFAIK. You can just write a more explicit sub-setting function `pp_hour[[j+1]] <- abind( lapply( pp, function(x) x[,,t_aux]))` – MrFlick Aug 09 '16 at 18:29
  • @MrFlick Seems to work okay for base arrays: `pp = list( array(33, dim=c(1,1,1,7)) ); lapply(pp, \`[\`, , , , 2)` – Frank Aug 09 '16 at 18:32
  • 1
    Cool @Frank. Did not know that. Must be a difference in the custom `[` function for `ff`. I don't really use that package so I probably won't dig into it any further. – MrFlick Aug 09 '16 at 18:33
  • Need code to create minimal example using `ff` functions. (Like MrFlick I don't use ff. Have enough RAM.) Sometimes you can use TRUE in a positional argument to act as a placeholder. Simple example: `array(1:27, c(3,3,3) )[ TRUE,TRUE , 2]` – IRTFM Aug 09 '16 at 20:24
  • So far I've solved it using a for loop, but is not as elegant and compact as using lapply. – JulianGiles Aug 10 '16 at 19:18

0 Answers0