1

I'm using the ff package in RStudio, which is running on a Windows server in my department. I'm using it to work with some large datasets, which I'm also storing on a network drive. I have confirmed that I have full read/write access to the drive in Windows, but when I use any functions in the ff package that directly writes a flat file to disk (such as ff() or as.ff() ), I get a "no diskspace" error.

However, if I write the data to another drive and then move it over to the network drive using pattern() it works fine. I only have this issue with the ff package. Other functions from base such as save() and write.csv() work fine. Is there a permissions issue that is specific to the ff package? I've included a minimal working example below.

> library(ff)
>
> # Set ff temporary directory to Network Drive
> options(fftempdir = "\\\\fs01/analysis/tmp/")
>
> test <- 1:10
> test.ff <- ff(test)
Error in ff(test) : no diskspace
>
> # But if I write it to another drive and then move it over, it works fine
>
> # Set the ff temporary directory to another drive
> options(fftempdir = "R:/analysis/tmp/")
> test.ff <- ff(test)
>
> physical(test.ff)$filename
[1] "R:/analysis/tmp/ff32081b6926e5.ff"
>
> pattern(test.ff) <- "\\\\fs01/analysis/tmp/"
>
> physical(test.ff)$filename
[1] "\\\\fs01/analysis/tmp/3208208747f3.ff"

And here is the software version information for R:

R version 3.1.2 (2014-10-31) -- "Pumpkin Helmet"
Copyright (C) 2014 The R Foundation for Statistical Computing
Platform: x86_64-w64-mingw32/x64 (64-bit)
thetanman6
  • 23
  • 3
  • The message says that your diskspace is full. Is your disk full? Does save(test, file = "\\\\fs01/analysis/tmp/test.RData") work or does it also fail because of your disk being full? –  Mar 12 '15 at 08:28
  • @jwijffels No, the disk is not full. There is ~500GB available. `save(test, file="\\\\fs01/analysis/tmp/test.RData")` and `write.csv(test, file=\\\\fs01/analysis/tmp/test.csv")` both work fine, as does creating the file elsewhere and moving it over using `pattern( )` – thetanman6 Mar 12 '15 at 13:37
  • I have worked with a similar setup using a network drive to store ff files but never encountered that problem. Only when my drive was full. –  Mar 12 '15 at 19:43

1 Answers1

1

Posting this here for future reference. We believe there is some kind of issue with resolving the server address within the ff sourcecode, maybe a bug. We ran into the exact same issue as you did - it was resolved by setting up a network drive named "x" for the remote location and then using the address

x:/yourfolder

instead of

\\servername/yourfolder

We also noticed that the ff file on disk is allocated nonetheless there is the error "no diskspace" when using something like

ff.test <- ff(vmode="double",filename="\\servername/youfolder/ff.test",dim=c(1000,1000))

and after sending the same line again, the ff object is created without errors.

tscherg
  • 1,032
  • 8
  • 22
  • Thanks for following up on this. I ended up stumbling upon this as well, but only through much trial and error. – thetanman6 Mar 26 '17 at 20:34