Stack Overflow
Stack Overflow

Questions tagged [ff]

An R package that provides memory-efficient storage of large data on disk and fast access functions

The ff package provides data structures that are stored on disk but behave (almost) as if they were in RAM by transparently mapping only a section (pagesize) in main memory.

More information:

165 questions
0
votes
0 answers

R bigglm ffdf returns size-related error

I am trying to run a logistic regression model on a ffdf using bigglm.ffdf from ffbase but I get an error: x1 = as.ff(sample(0:5, 400, replace=T)) x2 = as.ff(sample(c(rep(0, 98), rep(1, 2)), 400, replace=T) ) df2 = ffdf(x1, x2) mod = bigglm.ffdf(x2…
Audrey
  • 212
  • 4
  • 15
0
votes
1 answer

Linear regression using ff package

I am using ff package to do linear regression in R. Here tt is as.ffdf object. The code goes as follows: > fit <- lm(ADA ~ DUMMY + NLEAD + BIG4 + LOGMKT + LEV + ROA + ROAL + LOSS + + CFO + BTM + GROWTH + ALTMAN + ABSACCRL + + …
Sumit
  • 2,242
  • 4
  • 25
  • 43
0
votes
1 answer

Operations using ffdfwith in R

I am using ff and R because I have a huge dataset (around 16 GB) to work with. As a test case, I got the file to read around 1M records and wrote it out as a ff database. system.time(te3 <- read.csv.ffdf(file="testdata.csv", sep = ",", header=TRUE,…
Krishnan
  • 1,265
  • 2
  • 13
  • 24
0
votes
1 answer

R with ff and FSelector package

I have a 1360x92735 csv dataset and I have to reduce dimensionality using FSelector package for R (information.gain()), but it requests a lot of ram. My question is, can I use the ff package in combination with FSelector? If yes, how? p.s. I have…
Descanso7
  • 55
  • 1
  • 7
0
votes
1 answer

Split Date to Day, Month and Year for ffdf Data in R

I'm using R's ff package with ffdf objects named MyData, (dim=c(10819740,16)). I'm trying to split the variable Date into Day, Month and Year and add these 3 variables into ffdf existing data MyData. For instance: My Date column named…
MizaUnic
  • 79
  • 11
0
votes
1 answer

Save big data file in R to be loaded afterwards in Matlab

I have created a 300000 x 7 numeric matrix in R and I want to work with it in both R and Matlab. However, I'm not able to create a file well readeable for Matlab. When using the command save(), with file=xx.csv, it recognizes 5 columns instead; with…
user3032330
  • 27
  • 5
0
votes
1 answer

How to load a big csv using ff package?

I'm new at R. I have a big csv around 2.5 gb with 3 columns and 40 million rows. The datatypes of the column are interger, datetime and char. How to load this csv using ff package?
user3094606
  • 1
0
votes
1 answer

How to cast ffdf to ff

I am using the read.table.ffdf function from the ff package to read a gigantic matrix of numbers. How do I then cast this resulting ffdf object to an ff object without first converting to RAM?
user2763361
  • 3,789
  • 11
  • 45
  • 81
0
votes
0 answers

Using FFDFs, linearly apportion amounts by month

This is a follow up question to Linearly apportion amounts by month. Firstly, many thanks to Señor O, Michele and G. Grothendieck for their help with the initial question. Now, onto my question. Please consider the following…
Tyler Durden
  • 303
  • 5
  • 12
0
votes
1 answer

How to use the list feature in ffload to selectively load an ffdf?

How do I use the list feature in ffloadproperly to selectively load a part of an ffdf? I have read numerous times that it is possible to selectively load parts of an ff dataframe, but can't seem to get it working myself. My issue is that I'm trying…
bibzzzz
  • 193
  • 1
  • 10
0
votes
1 answer

ffdfdply function crashes R and is very slow

learning how to compute tasks in R for large data sets (more than 1 or 2 GB), I am trying to use ff package and ffdfdply function. (See this link on how to use ffdfdply: R language: problems computing "group by" or split with ff package ) My data…
Miguel Vazq
  • 1,459
  • 2
  • 15
  • 21
-1
votes
1 answer

Removing rows in list (with ff)

I have a data set where I want to remove every row in which Dataset$a does not have the value "Right". Dataset$a is a list with three diffrent objects "Right", "Wrong1" and "Wrong2". I tried to do this by using the code: Dataset$a <-…
Erik
  • 73
  • 1
  • 8
-1
votes
1 answer

Loading ff object after ff files were moved

I had a CNSet object created by the crlmm package that was stored using ff package. I saved it as an RData file (using the save function, not ffsave). And then I had to move my ff files to another location. Then I tried to load the object using the…
user890739
  • 702
  • 1
  • 13
  • 33
-1
votes
1 answer

R Logistic regression on ffdf objects

I have built a logistic regression model using the glm function from the stats package. I now would like to predict the outcome of this model on a large number of values, stored in a "ffdf" object (see ff package), however I do not find how to…
cho7tom
  • 1,030
  • 2
  • 13
  • 30
-2
votes
1 answer

Unable to use `library ("ff")` in R 3.4.2

I am not able to use library ("ff"). I don't know which library package ff comes under. I am using R 3.4.2.
Athul Vasan
  • 1
1 2 3
…
10
11