Limit lines of a txt file

Question

I have a text file where I save a line every 30 minutes with the WriteLineAsync method. When the file become too large, if I try to read it, the application crashes. I thought that I could limit the lines writable on the file so when I add a new line, the oldest line will be deleted. How can I do that?

edit: I read the file with the following code:

StorageFile MyFile = await ApplicationData.Current.LocalFolder.GetFileAsync("LogFile.txt");
string nextLine;
using (StreamReader reader = new StreamReader(await MyFile.OpenStreamForReadAsync()))
{
    while ((nextLine = await reader.ReadLineAsync()) != null)
    {
        TextLog.Text += nextLine + "\n";
    }
}

I've tried to debug it and I don't get any exception from the reading code. Maybe the problem is that I try to put all that text in a textblock.

You have to rewrite the whole file. So i would do it only occasionally. But why do you read the whole file at all if the app crashes? You could simply read less. — Tim Schmelter, Jan 29 '16 at 10:48
Or you could fix the code that crashes. What does "the file become too large" even mean? Is it *over* 2GB? Is there an exception? Where does it occur? Where is the code? Simply calling `WriteLineAsync` shouldn't fail, no matter the file's size. — Panagiotis Kanavos, Jan 29 '16 at 10:49
PS, it almost sounds like you are trying to log. Instead of reinventing the wheel, use one of the many logging libraries like log4net, NLog and with a rolling file option — Panagiotis Kanavos, Jan 29 '16 at 10:50

Lasse V. Karlsen · Answer 1 · 2016-01-29T10:52:32.403

If you really need to do this to a text file then you will have to do one of the two following:

Read the entire file into memory, chop off the unwanted part, and write it back out
Read the file line by line, ignoring the first X lines to get below your threshold, writing out lines you want to a temporary file. Once you've filtered all the lines you want, move the temporary file on top of the existing file.

There are also other alternatives:

Use a file format / data structure that supports arbitrary deletes, like a database.
Use rolling logs, starting a new file when the existing one becomes too big. There are plenty of existing logging libraries that will do this for you out of the box with just a bit of configuration.
Stop reading the whole file, only read the end of it. Granted, this has both upsides and downsides. The upside is that you can keep the file as big as you want. The downside is that if you need to guarantee to read the last N lines then it takes a bit more work to ensure that.

3. split it into multiple (history-)files 4. read less and not the whole file — Tim Schmelter, Jan 29 '16 at 10:51
Better yet, do none of them. Add a logging library and let it do the rolling — Panagiotis Kanavos, Jan 29 '16 at 10:51

score 1 · Answer 2 · answered Jan 29 '16 at 14:44

If you know the maximum length of a line you need to log and you don't mind trimming null from your log strings, the following solution might work for you.

It's a simple idea where you create an empty file knowing the maximum size and write the maximum line length of bytes at the current position each write. You simply loop back to the start of the file when you reach the end - meaning you will overwrite the first and oldest entry on the next write.

class MyWriter : IDisposable
{
    BinaryWriter _writer;
    readonly int _maxLineLength, _maxLines, _size;

    public MyWriter(string path, int maxLineLength, int maxLines)
    {
        _maxLineLength = maxLineLength;
        _maxLines = maxLines;
        _size = _maxLineLength * _maxLines;

        _writer = new BinaryWriter(File.Create(path));
        _writer.BaseStream.SetLength(_size);
    }

    public void Write(string str)
    {
        if (str.Length > _maxLineLength) throw new ArgumentOutOfRangeException();
        // Write the string to the current poisition in the stream.
        // Pad the rest of the line with null.
        _writer.Write(str.PadRight(_maxLineLength, '\0').ToCharArray());
        // If the end of the stream is reached, simply loop back to the start.
        // The oldest entry will then be overwritten next.
        if (_writer.BaseStream.Position == _size)
            _writer.Seek(0, SeekOrigin.Begin);
    }

    public void Dispose()
    {
        if(_writer != null)
            _writer.Dispose();
    }
}

Potentially used as:

using(var writer = new MyWriter("MyFile.txt", 200, 100))
{
    writer.Write("Hello World!");
}

Limit lines of a txt file

2 Answers2