DotNetZip why is the last line always truncated

Question

every zip file has a truncated last row. I output the row tot he console and it's not truncated, why would the last line of the compressed file be truncated?

using (SqlConnection conn = new SqlConnection(ConfigurationManager.AppSettings["connstring"]))
        {
          conn.Open();
          using (SqlCommand cmd = new SqlCommand())
          {
            cmd.Connection = conn;
            cmd.CommandType = CommandType.StoredProcedure;
            cmd.CommandText = "final_out";
            cmd.Parameters.Add("@jobid", SqlDbType.VarChar).Value = jobid;
            cmd.Parameters.Add("@filename", SqlDbType.VarChar).Value = filename;
            using (SqlDataReader rdr = cmd.ExecuteReader())
            {
              using (MemoryStream ms = new MemoryStream())
              {
                using (StreamWriter sw = new StreamWriter(ms))
                {
                  while (rdr.Read())
                  {
                    string nextval = rdr[0].ToString();
                    sw.WriteLine(nextval);
                    Console.WriteLine(nextval);
                  }
                  ms.Flush();
                  ms.Position = 0;
                  using (ZipFile zf = new ZipFile(Path.Combine(Path.GetDirectoryName(finalPath), Path.GetFileNameWithoutExtension(finalPath) + ".zip")))
                  {
                    zf.AddEntry(Path.GetFileName(finalPath), ms);
                    zf.Save();
                  }
                }
              }
            }
          }
        }

and I guess it's not fair to blame dotnetzip on this because the root of the problem is the streamwriter and memory stream not playing well together on the 'WriteLine' method. My assumption was that 'WriteLine' would Write the entire line to the stream which is not the case. — Chris Hayes, Jan 21 '14 at 19:44

Lasse V. Karlsen · Accepted Answer · 2014-01-21T19:31:20.410

3

Because you're not flushing the StreamWriter.

You're flushing the MemoryStream, which is a no-op.

The StreamWriter class keeps a buffer internally, which you have not flushed, so the last few lines and/or characters will still be sitting in this buffer. If you click the link and look at the documentation you can see that the constructor documentation says such things as:

Initializes a new instance of the StreamWriter class for the specified stream by using UTF-8 encoding and the default buffer size.

(my emphasis)

Replace this:

ms.Flush();

with this:

sw.Flush();

Here's a simple LINQPad program that demonstrates:

void Main()
{
    using (var ms = new MemoryStream())
    using (var sw = new StreamWriter(ms, Encoding.UTF8))
    {
        for (int index = 1; index <= 200; index++)
            sw.WriteLine("Line #" + index);

        ms.Flush();
        ms.Position = 0;

        Encoding.UTF8.GetString(ms.ToArray()).Dump();
    }
}

Output:

Line #1
Line #2
...
Line #196

(missing is line 197 through 200.)

After replacing ms.Flush(); with sw.Flush(); and running again the missing lines are present.

edited Jan 21 '14 at 19:31

answered Jan 21 '14 at 19:25

Lasse V. Karlsen

380,855
102
628
825

that did the trick, didn't realize I had to flush the streamwriter to force the last segment to write. must be a buffer size issue? – Chris Hayes Jan 21 '14 at 19:40
No, it's the same issue you will have with all buffered output, if you don't flush the buffer, you'll risk losing the last bit. – Lasse V. Karlsen Jan 21 '14 at 20:33
I'm just curious, would disposing the stream writer before writing the zip solve this too? – Chris Hayes Jan 21 '14 at 23:15
1

Well, yes and no. When you dispose the streamwriter, it will in fact dispose the underlying stream, so in general, this is probably not a good idea. *However*, since the underlying stream is a memory stream, disposing of that won't actually do anything so in this *particular* case, it would do the same trick. But in general, I would advise you to not dispose of the streamwriter, and instead flush it. – Lasse V. Karlsen Jan 22 '14 at 07:22

DotNetZip why is the last line always truncated

1 Answers1