Zipping huge amount of files ( zip outcome over 1GB )

Question

I've wrote a small app that compresses files and deletes them using DotNetZip,
it achieved compressing files to 1.4 GB zips
probably zips larger then this it's crashing

here is the stacktrace I got:

   at Ionic.Zip.ZipFile.Save()
   at Ionic.Zip.ZipFile.Save(String fileName)
   at CustomFileCompressor.Program.ParallelCompress(List`1 relevantFiles, Int32
zipYear, Int32 zipMonth, String outputFile) in d:\C#\CustomFileCompressor\Custom
FileCompressor\CustomFileCompressor\Program.cs:line 135
   at CustomFileCompressor.Program.Main(String[] args) in d:\C#\CustomFileCompre
ssor\CustomFileCompressor\CustomFileCompressor\Program.cs:line 78

here is the compression code, I tried to split it to 1GB Zip files
line 78 is the one call the function below

line 78:   ParallelCompress(FilesInMonth, zipYear, zipMonth, outFile);


static void  ParallelCompress(List<FileInfo> relevantFiles, int zipYear, int zipMonth, string outputFile)
    {
        List<string> filesToZip = relevantFiles.Select(fi => fi.FullName).ToList();
        using (ZipFile zip = new ZipFile())
        {
            string outputfile = string.Format(outputFile, zipYear, zipMonth);
            zip.MaxOutputSegmentSize = 1024 * 1024;
            zip.AddFiles(filesToZip, true, "");

            zip.Save(outputfile);
        }
    }

Umm... it looks like there is some information missing from the start of the stack trace. — Andrew Morton, Mar 16 '17 at 20:19
Cutting off the exception removes the useful part of the stack trace, you know? — TZHX, Mar 16 '17 at 20:19
thats the stack trace, isn't it enough ? `catch (Exception ex) {Console.WriteLine(ex.StackTrace)}` — Light_User, Mar 16 '17 at 20:22
No. You need to say what the exception was that was thrown. It probably tells in more detail what the problem is, not just where. — TZHX, Mar 16 '17 at 21:03
@TZHX yes... apparently not enough , `Ionic.Zip.ZipException: The number of entries is 65535 or greater` I think this explains a lot I'll try to use the `UseZip64WhenSaving` - hope it will help — Light_User, Mar 16 '17 at 21:05

score 0 · Answer 1 · answered Mar 16 '17 at 22:45

issue solved
apparently I was missing a configuration
after adding

zip.UseZip64WhenSaving = Zip64Option.Always;

changing the function to the below:

static void  ParallelCompress(List<FileInfo> relevantFiles, int zipYear, int zipMonth, string outputFile)
    {
        List<string> filesToZip = relevantFiles.Select(fi => fi.FullName).ToList();
        using (ZipFile zip = new ZipFile())
        {
            string outputfile = string.Format(outputFile, zipYear, zipMonth);
            zip.ParallelDeflateThreshold = 0;
            zip.UseZip64WhenSaving = Zip64Option.Always;
            zip.MaxOutputSegmentSize = 1024 * 1024 * 1024;
            zip.AddFiles(filesToZip, true, "");

            zip.Save(outputfile);
        }
    }

it solved the zipping
thanks for the useful comments

Zipping huge amount of files ( zip outcome over 1GB )

1 Answers1