Why does my CSV file gets corrupted when uploaded to Azure Blob Storage?

Question

My Azure Function creates a CSV file, gzip it and then upload it to Azure Blob Storage with this code:

var blobServiceClient = new BlobServiceClient("...");
var containerClient = blobServiceClient.GetBlobContainerClient("...");
var config = new CsvConfiguration(CultureInfo.CurrentCulture) { Delimiter = ";", Encoding = Encoding.UTF8 };

var list = new List<FakeModel>
{
   new FakeModel { Field1 = "A", Field2 = "B" },
   new FakeModel { Field1 = "C", Field2 = "D" }
};

var now = DateTime.Now;
var csvFileName = Path.Combine(Path.GetTempPath(), now.Hour + "_" + now.Minute + "_" + now.Second + ".csv");

using (var writer = new StreamWriter(csvFileName))
using (var csv = new CsvWriter(writer, config))
{
   await csv.WriteRecordsAsync(list);
   await csv.FlushAsync();
}

var gzFileName = csvFileName.Replace(".csv", ".csv.gz");

var bytes = await File.ReadAllBytesAsync(csvFileName);

using (var fs = new FileStream(gzFileName, FileMode.CreateNew))
using (var zipStream = new GZipStream(fs, CompressionMode.Compress, true))
{
   zipStream.Write(bytes, 0, bytes.Length);
   await zipStream.FlushAsync();

   fs.Position = 0;

   var blockBlob = containerClient.GetBlockBlobClient(gzFileName.Substring(csvFileName.LastIndexOf('\\') + 1));
   await blockBlob.UploadAsync(fs);
}

File.Delete(csvFileName);
File.Delete(gzFileName);

Everything works fine and the *.gz file is uploaded to Azure, but when I download it from Azure and open it locally with 7z to extract it, 7z gives my the following error:

Locally, the two files and created correctly and I can open both the *.csv and the *.csv.gz one.

Can you check the size of the uploaded file and compare it with the local one? Also, how are you downloading the file? — Gaurav Mantri, Jun 11 '21 at 15:44

score 0 · Answer 1 · answered Jun 11 '21 at 15:54

UploadAsync() accepts a BlobUploadOptions argument. Try setting it to something like this:

var opts = new BlobUploadOptions {
    HttpHeaders = new BlobHttpHeaders {
        ContentType = "text/csv;charset=utf-8", // or try utf-16 here
        ContentEncoding = "gzip"
    }
};

Why does my CSV file gets corrupted when uploaded to Azure Blob Storage?

1 Answers1