0

I have a large number of .Rdata files that I am loading into R to perform some analyses on, approximately 65 500. Each file represent a statistical model in order to perform an APM analyese. Since I named each file after the model it holds, some of them have quite long file names, the longest being 190 characters (including the path name).

I load the .Rdata files in by reading all the file names within a folder, then load them using a for loop (simplified code below): 

wrkspce = list.files(path = "Workspace/", pattern = "*.Rdata")

for(i in 1:length(wrkspce)){
    load(paste0("Workspace/", wrkspce[i], sep = ""))
}

However, as it goes through the files, it always fails saying a file is missing. I have checked to make sure the file wasn't empty or the computer mistakenly thought it existed - but the file does exist.

I have read elsewhere that there is a 260 character limit to paths in other languages - is there a similar limit within R? The longest path I have is 190, so well below the 260 limit I had read of elsewhere.

Does anyone have any suggestions as to why this may be happening?

UPDATE

The specific error I am getting is:

Error in readChar(con, 5L, useBytes = TRUE) : cannot open the connection
In addition: Warning message:
In readChar(con, 5L, useBytes = TRUE) :
  cannot open compressed file 'Workspace/Replacement_NTRemoved/Replacement_NTRemoved_LT/Workspace/Replacement_LogArea+LogIsolation+AbsLat+LogElevation+Age3+LogRainfall+Makatea+Dust+Tephra2+Tephra3+Dry+Tree+C+I.Rdata', probable reason 'No such file or directory'

And I can confirm that the file does exist, and can load that file by double clicking it on its own. But it will not open in the loop.

SamPassmore
  • 1,221
  • 1
  • 12
  • 32
  • is it the longest path that is missing? – jeborsel Mar 29 '15 at 21:56
  • No, the iteration stops before then. The path length of that file is 184. I get the same error for both this file and the file with the longest path. – SamPassmore Mar 29 '15 at 22:01
  • If you try to run your load() function just for that file, does that work? so load(paste0("Workspace/", wrkspce[i], sep = "")) where i is the file that does not work in the loop – jeborsel Mar 29 '15 at 22:03
  • No, I get the same error. I also get that error if i write the whole path in manually, rather than using paste. This is for both the file that breaks the loop and the longest path file. – SamPassmore Mar 29 '15 at 22:13
  • I should note that ~28 000 workspaces are loaded before this, so it is not like the loop is not working at all. Is there a maximum length of lists? – SamPassmore Mar 29 '15 at 22:14
  • hard to reproduce... all working fine here, is your working directory a local path? ... (btw if you use function paste0 () , there is no need to put sep="", actually paste0(...) is equivalent to paste(..., sep = "") ) – jeborsel Mar 29 '15 at 23:12
  • I managed to solve it by just moving my data to the desktop and working from their. Which isn't ideal - but it works. Thanks for your help though! – SamPassmore Mar 29 '15 at 23:41

0 Answers0