Streamwriter is cutting off my last couple of lines sometimes in the middle of a line?

Question

Here is my code. :

FileStream fileStreamRead = new FileStream(pathAndFileName, FileMode.OpenOrCreate, FileAccess.Read, FileShare.None);
FileStream fileStreamWrite = new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);

                StreamWriter sw = new StreamWriter(fileStreamWrite);

                int readIndex = 0;
                using (StreamReader sr = new StreamReader(fileStreamRead))
                {
                    while (!sr.EndOfStream) {
                        Console.WriteLine("eof" + sr.EndOfStream);
                        readIndex++;
                        Console.WriteLine(readIndex);
                        string currentRecord = "";
                        currentRecord = sr.ReadLine();
                        if (currentRecord.Trim() != "")
                        {
                            Console.WriteLine("Writing " + readIndex);
                            sw.WriteLine(currentRecord);
                        }
                        else {
                            Console.WriteLine("*******************************************spaces ***********************");
                        }
                    }

It is cutting off 2 lines with one test file and half a line, and then 1 line and half a line with the other test file I am running it against.

I am not a streamreader/writer expert you can probably see.

Any ideas or suggestions would be greatly appreciated as this is driving me batty. I am sure it is me using these incorrectly.

score 25 · Accepted Answer · answered Oct 04 '12 at 21:06

25

You are missing Flush/Close or simply using for your writer.

using(FileStream fileStreamWrite = 
  new FileStream(reProcessedFile, FileMode.OpenOrCreate, FileAccess.Write, FileShare.None);
{
  using(StreamWriter sw = new StreamWriter(fileStreamWrite))
  {
   // .... write everything here
  }
}

answered Oct 04 '12 at 21:06

Alexei Levenkov

98,904
14
127
179

3

@Tigran - most Writer and Stream classes have internal buffers of limited size, so periodically they commit data to underlying storage. As result some data is stored, but last chunk could be lost if not flushed/closed. – Alexei Levenkov Oct 04 '12 at 22:24

score 11 · Answer 2 · answered Oct 04 '12 at 21:08

11

Right after the closing brace of the using statement, do this:

sw.Flush();
sw.Close();

There, that should do it.

answered Oct 04 '12 at 21:08

code4life

15,655
7
50
82

came across this answer, i had basically the same question. For my situation, this helped a lot. Thanks. – wondergoat77 Aug 24 '13 at 00:56
Dispose() which calls Close() which calls Flush() – Abdullah Tahan Aug 26 '22 at 14:51

zmbq · Answer 3 · 2012-10-04T21:13:05.907

4

You need to Flush your StreamWriter. A StreamWriter has a buffer, and it writes to disk only when the buffer is full. By flushing at the end you make sure all the text in the buffer is written to the disk.

edited Oct 04 '12 at 21:13

answered Oct 04 '12 at 21:06

zmbq

38,013
14
101
171

@UserSmith, once after all writes, also missing using on Files (see my answer) – Alexei Levenkov Oct 04 '12 at 21:09

score 2 · Answer 4 · answered Oct 04 '12 at 21:22

In addition to other answers (use using, and/or flush/close), would say that they do not actually respond to the question: "why it may cut several lines."

I have an idea on subject that it is related to a fact that you use StreamReader and call EndOfStream twice: in a while loop header, and another inside it.

The only possible way of understanding if the stream ends is try to read some data from it. So I suspect EnfOfStream does it, and reading it twice, may create a problem in stream processing.

To resolve an issue:

Or use simple TextReader, considering that you are reading text file (seems to me)
Or change your logic to call only once, so no more call to Console.WriteLine("eof" + sr.EndOfStream);
Or change your logic, so do not use EndOFStream at all, but read line by line till the line is null.

score 1 · Answer 5 · answered Oct 04 '12 at 21:34

You're not using StreamWriter properly. Also, since you're always reading lines, I would use a method that already does all that for you (and manages it properly).

using (var writer = new StreamWriter("path"))
{
   foreach(var line in File.ReadLines("path"))
   {
       if (string.IsNullOrWhiteSpace(line)) 
       { /**/ }
       else
       { /**/ }
   }
}

... or ...

/* do not call .ToArray or something that will evaluate this _here_, let WriteAllLines do that */
var lines = File.ReadLines("path")
                .Select(line => string.IsNullOrWhiteSpace(line) ? Stars : line);

var encoding = Encoding.ASCII; // whatever is appropriate for you.
File.WriteAllLines("path", lines, encoding);

Streamwriter is cutting off my last couple of lines sometimes in the middle of a line?

5 Answers5

Linked

Related