0

This is a follow up question to Linearly apportion amounts by month.

Firstly, many thanks to Señor O, Michele and G. Grothendieck for their help with the initial question.

Now, onto my question. Please consider the following FFDF:

require(ffbase)
start <- c(as.Date("2013-01-01"), as.Date("2013-02-01"), as.Date("2013-04-01"), as.Date("2013-04-16"), as.Date("2013-05-16"))
end <- c(as.Date("2013-01-31"), as.Date("2013-03-31"), as.Date("2013-04-15"), as.Date("2013-05-15"), as.Date("2013-05-31"))
amount <- c(100, 200, 50, 100, 50)
id <- c("M1","M2","M3","M4","M5")
df <- data.frame(start,end,amount,id)
ffdf <- as.ffdf(df)

I would like to do the same as before, split out the amounts linearly by calendar month. But this time working with FFDF data structures.

The initial answer, using Map to "spit" the outputs of a custom explode function into an rbind call followed by an aggregation proved to be a very good learning experience. It did take two days to really work out how that solution worked but I got there in the end and did exactly what I was after.

When applying this learning to a real dataset, however, I ran into memory issues whereupon I discovered the ff package and this has been very useful.

To try and do the same thing with ffdfs I've attempted use of ffdfdply but found myself very lost and am turning to SO for some guidance.

Many Thanks!

Community
  • 1
  • 1
Tyler Durden
  • 303
  • 5
  • 12
  • 1
    I have added to the prior original answer at http://stackoverflow.com/questions/19321877/linearly-apportion-amounts-by-month/19322571#19322571 to use less memmory. Maybe with that you don't need ffdf? – G. Grothendieck Oct 15 '13 at 15:28
  • Thanks G. I've implemented your tweak to the custom explode function which brings the aggregation into the function itself. It's been over half an hour and it's running at 100% RAM usage (hasn't yet broken with a memory error). Your solution before was very helpful to learn about advanced manipulation of data frames. At the moment my knowledge of ffdfs is very basic and an illustrative example of doing manipulations such as these will really help my understand applying functions etc to ffdfs. – Tyler Durden Oct 15 '13 at 16:09
  • can you show what you have tried with ff and where you got stuck? –  Oct 16 '13 at 09:49
  • I have added yet another solution marked UPDATE 2 to the original post http://stackoverflow.com/questions/19321877/linearly-apportion-amounts-by-month/19322571#19322571 . This new one runs 10x faster on the data in that post in my test. – G. Grothendieck Oct 16 '13 at 13:20

0 Answers0