How to get the compressed size (ZipArchive ) of a FileStream block?

Question

I'm writing a little C# appx package editor (appx is basically a zip file containing a bunch of XML metadata files).

In order to make a valid appx file, I need to create a block map file (XML) that contains for each file two attributes : hash and size as explained here (https://learn.microsoft.com/en-us/uwp/schemas/blockmapschema/element-block)

Hash represent a 64kb uncompressed chunk of a file. Size represent the size of that chunk after being compressed (deflate algorithm). Here is what I wrote so far as proof of concept :

using System;
using System.IO;
using System.IO.Compression;
using System.Linq;

namespace StreamTest
{
    class Program
    {
        static void Main(string[] args)
        {
            using (var srcFile = File.OpenRead(@"C:\Test\sample.txt"))
            {
                ZipAndHash(srcFile);
            }

        Console.ReadLine();
    }

    static void ZipAndHash(Stream inStream)
    {
        const int blockSize = 65536; //64KB
        var uncompressedBuffer = new byte[blockSize];
        int bytesRead;
        int totalBytesRead = 0;

        //Create a ZIP file
        using (FileStream zipStream = new FileStream(@"C:\Test\test.zip", FileMode.Create))
        {
            using (ZipArchive zipArchive = new ZipArchive(zipStream, ZipArchiveMode.Create))
            {
                using (BinaryWriter zipWriter = new BinaryWriter(zipArchive.CreateEntry("test.txt").Open()))
                {
                    //Read stream with a 64kb buffer
                    while ((bytesRead = inStream.Read(uncompressedBuffer, 0, uncompressedBuffer.Length)) > 0)
                    {
                        totalBytesRead = totalBytesRead + bytesRead;

                        //Compress current block to the Zip entry 
                        if (uncompressedBuffer.Length == bytesRead)
                        {
                            //Hash each 64kb block before compression
                            hashBlock(uncompressedBuffer);

                            //Compress
                            zipWriter.Write(uncompressedBuffer);
                        }
                        else
                        {
                            //Hash remaining bytes and compress
                            hashBlock(uncompressedBuffer.Take(bytesRead).ToArray());
                            zipWriter.Write(uncompressedBuffer.Take(bytesRead).ToArray());
                        }
                    }

                    //How to obtain the size of the compressed block after invoking zipWriter.Write() ?

                    Console.WriteLine($"total bytes : {totalBytesRead}");
                }
            }
        }

    }

    private static void hashBlock(byte[] uncompressedBuffer)
    {
        // hash the block
    }
  }
}

I can easily get the hash attribute by using a 64kb buffer while reading a stream, my question is :

How do I obtain the compressed size of each 64kb block after using zipWrite.Write(), is it even possible with System.IO.Compression or should I use something else ?

Answer 1

If your problem still actual, for creating container you can use 2 approach:

1. Managed OPC Packaging APIs, which provide support for applications that produce or consume Open Packaging Conventions compliant files, called packages, and develop all specific things by yourself (general description is here: https://msdn.microsoft.com/en-us/library/windows/desktop/dd371623(v=vs.85).aspx)

For getting blocks compression size on-the-fly you can use DeflateStream and MemoryStream like below:

      private static long getCompressSize(byte[] input)
    {

        long length = 0;

        using (MemoryStream compressedStream = new MemoryStream())
        {
            compressedStream.Position = 0;

            using (DeflateStream compressionStream = new DeflateStream(compressedStream, CompressionLevel.Optimal, true))
            {
                compressionStream.Write(input, 0, input.Length);

            }


            length = compressedStream.Length;

        }
        Logger.WriteLine("input length:" + input.Length + " compressed stream: " + length);
        return length;
    }

2. C++ API for Appx containers (but in this case Project should be rewritten using C++ or additional library should be written and imported to C# project). The main advantage is that it already has methods for create all needed package parts. General description is here (https://msdn.microsoft.com/en-us/library/windows/desktop/hh446766(v=vs.85).aspx)

Solution for getting block size and Hash:

The IAppxBlockMapBlock interface provides a read-only object that represents an individual block within a file contained in the block map file (AppxBlockMap.xml) for the App package. The IAppxBlockMapFile::GetBlocks method is used to return an enumerator for traversing and retrieving the individual blocks of a file listed in the package block map.

The IAppxBlockMapBlock interface inherits from the IUnknown interface and has these methods:

GetCompressedSize - Retrieves compressed size of the block.

GetHash - Retrieves the hash value of the block.