Stack Overflow
Stack Overflow

Questions tagged [ffbase]

Basic statistical functions for package ff

R package ffbase provides basic functionality to allow to do basic data operations on ff structures. This functionality is also available in the R base library and ffbase tries to provide the methods needed for ff structures such that switching between in-RAM objects and on-disk objects in ff is more straightforward.

More info:

64 questions
0
votes
1 answer

How to change an ff vector to a normal vector R

I have a function that accepts vectors as inputs. I have an ffdf named X, and would like to use columns of data as vectors for the function inputs. To take a column of data named "Mag" as a vector I would use the following: X[['Mag']] However this…
user3565975
  • 75
  • 2
  • 9
0
votes
0 answers

Writing a Loop using ffwhich function from ff package R

I have a very large ffdf named 'Scenarios' and I would like to sort one of the columns named 'Eventfreq' into bins based on two variables and then take the sum of the values within each BIN. The function below essentially does this for one set of…
user3565975
  • 75
  • 2
  • 9
0
votes
0 answers

Creating dummy variables in a FF object

I have an FFDF object that I'd like to create dummy variables in. Specifically, I have a separate vector of names and I want to make dummy variables if a name in that vector matches a name I have in a column vector in my FFDF. As an example,…
ModalBro
  • 544
  • 5
  • 25
0
votes
2 answers

How to overload the functions "[<-.ffdf" and "[.ffdf" from package ff (in R)?

I've experienced the write error from ff package. This answer suggests that the solution could be to overload the functions "[<-.ffdf" and "[.ffdf". Could somebody elaborate this in more details, or at least suggest what I need to figure out to…
H13
  • 1
  • 2
0
votes
0 answers

R bigglm ffdf returns size-related error

I am trying to run a logistic regression model on a ffdf using bigglm.ffdf from ffbase but I get an error: x1 = as.ff(sample(0:5, 400, replace=T)) x2 = as.ff(sample(c(rep(0, 98), rep(1, 2)), 400, replace=T) ) df2 = ffdf(x1, x2) mod = bigglm.ffdf(x2…
Audrey
  • 212
  • 4
  • 15
0
votes
1 answer

R ffbase save.ffdf error ( NA were not saved, because not found )

I have a large data.table (let's call it DT) in R which takes too long to save as csv (would be great to have a fast writer similar to fread). Therefore I am converting the data.table to a ffdf object: library(data.table) library(ffbase) ffiles…
user3340145
  • 157
  • 9
0
votes
1 answer

Operations using ffdfwith in R

I am using ff and R because I have a huge dataset (around 16 GB) to work with. As a test case, I got the file to read around 1M records and wrote it out as a ff database. system.time(te3 <- read.csv.ffdf(file="testdata.csv", sep = ",", header=TRUE,…
Krishnan
  • 1,265
  • 2
  • 13
  • 24
0
votes
0 answers

R ff inconsistency with ffwhich?

I am trying to subset an ffdf object by selective indexing of its first column and came across the following paradox (inconsistency?): # create an ffdf object: > df_ff=as.ffdf(data.frame( names=c('B','A','B','A','A' ),numbers=1:5)) # 1) Works…
Audrey
  • 212
  • 4
  • 15
0
votes
0 answers

ffbase package fails to load

I have just installed the ff and ffbase packages in R, and while library(ff) works, library(ffbase) returns the following error: > library(ffbase) Error in library.dynam(lib, package, package.lib) : DLL ‘fastmatch’ not found: maybe not installed…
Audrey
  • 212
  • 4
  • 15
0
votes
1 answer

Split Date to Day, Month and Year for ffdf Data in R

I'm using R's ff package with ffdf objects named MyData, (dim=c(10819740,16)). I'm trying to split the variable Date into Day, Month and Year and add these 3 variables into ffdf existing data MyData. For instance: My Date column named…
MizaUnic
  • 79
  • 11
0
votes
1 answer

Not creating a copy with ffload

I have a question about a function attribut from the ff package. I have an ff object in a specific directory I want to load. This is what I am using: setwd(~/Documents/Data) ffload("Object", overwrite=TRUE) With OVERWRITE=TRUE, a copy of the object…
Chika
  • 1,455
  • 2
  • 16
  • 24
0
votes
1 answer

Use a variable with a created name in a function

I don't know if what I try to do is possible! I have a matrix of values (Var_sim) and I convert it as an ff object. The name of this object is created with "paste". Now I want to use this object in the function ffload, this is my code: Variables[k]…
Douie
  • 201
  • 4
  • 11
0
votes
1 answer

Write a ff file

I am working with large matrix on R and I try to use the ff package to manipulate them. I have a loop which will create n (n can be size 1000) matrix called Var_sim (dim 25*7303) I want to save the matrix in the disk (for example in my documents)…
Douie
  • 201
  • 4
  • 11
0
votes
1 answer

aggregation using ffdfdply function in R

I tried aggregation on large dataset using 'ffbase' package using ffdfdply function in R. lets say I have three variables called Date,Item and sales. Here I want to aggregate the sales over Date and Item using sum function. Could you please guide me…
Chaitanya Krishna T
  • 11
  • 1
  • 5
0
votes
0 answers

How to fix save.ffdf on remote server

I can't save.ffdf or move.ffdf. Also, the deprecated ffdfsave is broken. require(ff);require(ffbase) x <- data.frame(matrix(rnorm(5*5),ncol=5)) x <- as.ffdf(x) save.ffdf(x,getwd()) I will get the following error: Error infilename<-.ff(tmp, value =…
user2763361
  • 3,789
  • 11
  • 45
  • 81
1 2 3
4
5