1

I have 7300 *.csv files in a temp directory. I want to zip these into a zip archive in R. I'm using the following code, which is taking FOREVER. Is there a way to do this faster, short of exiting R and using the WinZip program?

fileListing       = list.files( pattern = '*.csv' )
outZipFileName    = gsub( '.zip', '_TZflts.zip', zipName )
sapply(seq_along( fileListing),function(ii) zip( outZipFileName, fileListing[ii] ) )

Another problem is that the zip process in the code spawns tons of garbage files, besides the zip file and its csv contents.

Thank you.

BSL

Benjamin Levy
  • 333
  • 6
  • 19
  • you dont need to loop through the files.. you can add them all in one `zip` call. Oh actually, have I misunderstood.. do you want a separate zip files for each csv, or all csv's in one zip? – user20650 Jul 09 '16 at 18:38
  • No. One zip archive containing 7300 csv files. – Benjamin Levy Jul 09 '16 at 18:55

1 Answers1

3

You do not need to loop through the files, zip can take a vector of the files to be zipped: this should speed things up. From ?zip

files is : 'A character vector of recorded filepaths to be included.'

Example

# write some files to be zipped
for(i in 1:10) write.csv(mtcars, paste0("SOtemp", i, ".csv"))

# zip
zip("SOzip", files=list.files(pattern="SOtemp\\d"))

# remove files from this example
# file.remove(c("SOzip.zip", list.files(pattern="SOtemp\\d")))
user20650
  • 24,654
  • 5
  • 56
  • 91
  • Tried it with your command. Received warning message, 'running command '"zip" -r9X ', followed by a listing of all 7300 csv files. No zip file was created. – Benjamin Levy Jul 09 '16 at 19:41
  • did the small example in my answer work? If not, http://stackoverflow.com/questions/29129681/create-zip-file-error-running-command-had-status-127 is probably useful (although i would try it again straight after installing rtools, before setting paths etc) – user20650 Jul 09 '16 at 20:08
  • okay, I just tried it on a windows partition and I got the same message as you. I then followed the steps (including setting the path) in [HBat's amswer](http://stackoverflow.com/questions/29129681/create-zip-file-error-running-command-had-status-127?answertab=votes#tab-top) and it now works. – user20650 Jul 09 '16 at 20:23
  • Will try your link. I found that zip( 'filename.zip', files=fileListing[1 : 1100] ) worked without error. So, I infer that there is a limit to the number of files that R can zip at once (either total bytes to zip or limit on total files). I'm going to just add 1000 files at a time. – Benjamin Levy Jul 09 '16 at 20:47