Stack Overflow
Stack Overflow

Questions tagged [ffbase]

Basic statistical functions for package ff

R package ffbase provides basic functionality to allow to do basic data operations on ff structures. This functionality is also available in the R base library and ffbase tries to provide the methods needed for ff structures such that switching between in-RAM objects and on-disk objects in ff is more straightforward.

More info:

64 questions
1
vote
1 answer

Merging ffdf dataframes in R

I need an outer join of ffdf dataframes saved in a list. Have checked this, but it refers to a different problem. Example code for RAM objects: x1 = data.frame(name='a1', Ai=2, Ac=1, Bi=1) x2 = data.frame(name='a2', Ai=1, Bi=3, Bc=1, Ci=1) x3 =…
Audrey
  • 212
  • 4
  • 15
1
vote
2 answers

Combine factor levels in an ff object

I often categorise times into day/night time using cut(). Because cut() doesn't understand that clock times go around zero, I first divide the hours into three groups (night either side of day), and then merge the two "night" factor levels. This…
nacnudus
  • 6,328
  • 5
  • 33
  • 47
1
vote
2 answers

Merging and appending ffdf dataframes

I am trying to create an ffdf dataframe by merging and appending two existing ffdf dataframes. The ffdfs have different numbers of columns and different row numbers. I know that merge() performs only inner and left outer joins while ffdfappend()…
Rkook
  • 63
  • 1
  • 11
1
vote
1 answer

Loading ff / ffbase files into R session with load.ffdf

I have a large ffdf data frame saved to disk that I need to load into a fresh R session. When I run load.ffdf in the directory where the file is located, I get the following error message: load.ffdf("./ffdb") # Error in `filename<-.ff`(`*tmp*`,…
dynamo
  • 2,988
  • 5
  • 27
  • 35
0
votes
1 answer

Merging and appending a list of ffdf dataframes

I would like to read a vector of CSV files names as ffdf data frames and combine them into one big ffdf data frame. I have found solutions using other r packages; however, my issue is my data (combined) can reach 40GB which definitely needs to be…
ahmathelte
  • 559
  • 3
  • 15
0
votes
1 answer

how to speed up checking duplication for huge ffdf

I have a list of ffdf, it takes up about 76GB of RAM if it is loaded to RAM instead of using ff package. The following is their respective dim() > ffdfs |> sapply(dim) [,1] [,2] [,3] [,4] [,5] [,6] [,7] [1,]…
Chris LAM
  • 142
  • 1
  • 7
0
votes
1 answer

Basic example not working for ffwhich from the ffbase-package

I'm trying to use OHDSI:s version of the SelfControlledCaseSeries package, which utilizes the ff package to handle big data. But something is not working with the ffwhich function. Running the following example, provided in the ffwhich…
Oskar Gauffin
  • 35
  • 4
0
votes
1 answer

Kriging simulation using ff package

I'm trying to understand the way I can use the ff package to overcome the error "Error: cannot allocate vector of size 1.1 Mb" while using kriging/ gaussian simulation. I don't know how to change the input data. Is there any idea to help me do…
Mohammad
  • 67
  • 1
  • 9
0
votes
1 answer

How to subset ffdf by index?

I would like to subset an ffdf object by index, returning another ffdf object. The help file on subset.ffdf indicates that you can pass a range index (ri) object as an argument, but when I tried: data_subset <- subset.ffdf(data, ri(1, 1e5)) I got…
travis
  • 5
  • 3
0
votes
1 answer

ffbase: merge on columns X and Y and closest column Z

I would like to accomplish the following using ffdf: Merge on columns X and Y and closest Time and then merge on the closes column B. However,the procedure that I know in smaller samples involves using outer merges (as shown below). What is a way…
dleal
  • 2,244
  • 6
  • 27
  • 49
0
votes
0 answers

How to sum up columns in table.ff or how to convert it to doable form

What's the 'nature' of a table.ff object in r? dim of table.ff is N ULL, and typically it is used for frequency measures. I could not find any funtion to add all columns together in order to do some statistics on resultant ' numeric vector'. str of…
Rafal Rakowski
  • 11
  • 3
0
votes
2 answers

How to DROP columns from ffdf object ? (R)

Could I easily drop column of ffdf object ? library(ff);library(ffbase) irisdf=as.ffdf(iris) How to contain only Sepal.length and Species columns ?
Qbik
  • 5,885
  • 14
  • 62
  • 93
0
votes
1 answer

Assign value to variable in ff

I am working with a ff dataset (final_faf_data) because of the size of my database. I am trying to create a new variable dpy which has only only one value (365) for all rows. When I try the following cmd (similar to what I would do in a…
Krishnan
  • 1,265
  • 2
  • 13
  • 24
0
votes
1 answer

Subsetting ffdf in loop

I'm trying to subset a very large ffdf object in a loop using ffbase, but I'm getting the error message: Error in UseMethod("as.hi") : no applicable method for 'as.hi' applied to an object of class "NULL" I'm running this code on an ssh with large…
ostionalzax
  • 25
  • 3
0
votes
1 answer

R ffbase merge issues

I'm seeing some strange behavior when I use merge with ffdf and I was wondering if someone can explain to me why this is happening and how can I fix it. Here is sample for regular data frames: dfx<-data.frame(a=1: 3, b=4:6) > dfy<-data.frame(a=c(1,…
Henry80s
  • 37
  • 5
1 2
3
4 5