Zip S3 files using C#

Question

What I'm looking for is zip/compress S3 files without having them first downloaded to EFS or on a file system and then upload the zip file back to S3. Is there a C# way to achieve the same? I found the following post, but not sure its C# equivalent https://www.antstack.io/blog/create-zip-using-lambda-with-files-streamed-from-s3/

I've written following code to zip files from a MemoryStream

public static void CreateZip(string zipFileName, List<FileInfo> filesToZip)
   {
        //zipFileName is the final zip file name
        LambdaLogger.Log($"Zipping in progress for: {zipFileName}");
        using (MemoryStream zipMS = new MemoryStream())
        {
            using (ZipArchive zipArchive = new ZipArchive(zipMS, ZipArchiveMode.Create, true))
            {
                //loop through files to add
                foreach (var fileToZip in filesToZip)
                {
                    //read the file bytes
                    byte[] fileToZipBytes = File.ReadAllBytes(fileToZip.FullName);
                    ZipArchiveEntry zipFileEntry = zipArchive.CreateEntry(fileToZip.Name);
                    //add the file contents
                    using (Stream zipEntryStream = zipFileEntry.Open())
                    using (BinaryWriter zipFileBinary = new BinaryWriter(zipEntryStream))
                    {
                        zipFileBinary.Write(fileToZipBytes);
                    }
               }
            }
            using (FileStream finalZipFileStream = new FileStream(zipFileName, FileMode.Create))
            {
                zipMS.Seek(0, SeekOrigin.Begin);
                zipMS.CopyTo(finalZipFileStream);
            }
        }
   }

But problem is how to make it read file directly from S3 and upload the compressed file.

Does this answer your question? [How to zip files in Amazon s3 Bucket and get its URL](https://stackoverflow.com/questions/43275575/how-to-zip-files-in-amazon-s3-bucket-and-get-its-url) — Liam, Mar 02 '21 at 10:09
It seems like you just need to figure out how to download an S3 file into a `MemoryStream`. — Enigmativity, Mar 02 '21 at 10:10

Aniruddha · Accepted Answer · 2021-03-03T05:04:38.567

public static async Task CreateZipFile(List<List<KeyVersion>> keyVersions)
{

    using MemoryStream zipMS = new MemoryStream();
    using (ZipArchive zipArchive = new ZipArchive(zipMS, ZipArchiveMode.Create, true))
    {
        foreach (var key in keyVersions)
        {
            foreach (var fileToZip in key)
            {
                GetObjectRequest request = new GetObjectRequest
                {
                    BucketName = "dev-s3-zip-bucket",
                    Key = fileToZip.Key
                };
                using GetObjectResponse response = await s3client.GetObjectAsync(request);
                using Stream responseStream = response.ResponseStream;
                
                ZipArchiveEntry zipFileEntry = zipArchive.CreateEntry(fileToZip.Key);
                    
                //add the file contents
                using Stream zipEntryStream = zipFileEntry.Open();
                await responseStream.CopyToAsync(zipEntryStream);
         
            }
        }
        zipArchive.Dispose();
    }
        zipMS.Seek(0, SeekOrigin.Begin);
        var fileTxfr = new TransferUtility(s3client);
        await fileTxfr.UploadAsync(zipMS, "dev-s3-zip-bucket", "test.zip");
}

Zip S3 files using C#

1 Answers1