How do I use File.ReadAllBytes In chunks

Question

I am using this code

        string location1 = textBox2.Text;
        byte[] bytes = File.ReadAllBytes(location1);
        string text = (Convert.ToBase64String(bytes));
        richTextBox1.Text = text;

But when I use a file that is too big I get out of memory exception.

I want to use File.ReadAllBytes in chunks. I have seen code like this below

System.IO.FileStream fs = new System.IO.FileStream(textBox2.Text, System.IO.FileMode.Open);
byte[] buf = new byte[BUF_SIZE];
int bytesRead;

// Read the file one kilobyte at a time.
do
{
    bytesRead = fs.Read(buf, 0, BUF_SIZE);               
    // 'buf' contains the last 1024 bytes read of the file.

} while (bytesRead == BUF_SIZE);

fs.Close();

}

But I don't know how to actually convert the bytesRead into a byte array which I will convert to text.

EDIT: Answer found. Here is the code!

 private void button1_Click(object sender, EventArgs e)
    {
        const int MAX_BUFFER = 2048;
        byte[] Buffer = new byte[MAX_BUFFER];
        int BytesRead;
        using (System.IO.FileStream fileStream = new FileStream(textBox2.Text, FileMode.Open, FileAccess.Read))
            while ((BytesRead = fileStream.Read(Buffer, 0, MAX_BUFFER)) != 0)
            {
                string text = (Convert.ToBase64String(Buffer));
                textBox1.Text = text;

            }

}

To change the readable bytes which are in text format, create a new byte and make it equal (Convert.FromBase64String(Text)). Thanks everyone!

score 2 · Accepted Answer · edited Oct 02 '12 at 09:01

private void button1_Click(object sender, EventArgs e)
{
    const int MAX_BUFFER = 2048;
    byte[] Buffer = new byte[MAX_BUFFER];
    int BytesRead;
    using (System.IO.FileStream fileStream = new FileStream(textBox2.Text, FileMode.Open, FileAccess.Read))
        while ((BytesRead = fileStream.Read(Buffer, 0, MAX_BUFFER)) != 0)
        {
            string text = (Convert.ToBase64String(Buffer));
            textBox1.Text = text;

        }
}

Mare Infinitus · Answer 2 · 2012-07-22T07:55:33.490

The bytesRead is just the count of bytes read.

Here is some block reading

        var path = @"C:\Temp\file.blob";

        using (Stream f = new FileStream(path, FileMode.Open))
        {
            int offset = 0;
            long len = f.Length;
            byte[] buffer = new byte[len];

            int readLen = 100; // using chunks of 100 for default

            while (offset != len)
            {
                if (offset + readLen > len)
                {
                    readLen = (int) len - offset;
                }
                offset += f.Read(buffer, offset, readLen);
            }
        }

Now you have the bytes in the buffer and can convert them as you like.

for example inside the "using stream":

            string result = string.Empty;

            foreach (byte b in buffer)
            {
                result += Convert.ToChar(b);
            }

score 0 · Answer 3 · answered Jul 22 '12 at 02:10

0

No, the return value from Read() is how many bytes were read. The data is in the byte array buf that you are passing to Read(). You should try and understand code instead of just copy & pasting, then asking why it doesn't work. Even if you do, the comment says it right there!

answered Jul 22 '12 at 02:10

Jonathon Reinhart

132,704
33
254
328

I tried. Can you Help me Please? How do I convert all the bufs to one big array? – Layne3 Jul 22 '12 at 02:13
1

Why would you do that? That defeats the whole purpose of reading it in chunks! And that's exactly what ReadAllBytes does – Jonathon Reinhart Jul 22 '12 at 02:14
OK i get it, so each time it reads it in a chunk, where Do i convert it to a string and add it to a textbox. Like which part of code – Layne3 Jul 22 '12 at 02:17
I don't know what exactly it is you're trying to accomplish! If the file is larger than the amount of memory you have, you're going to run out of memory. Period. You should explain what exactly it is you're trying to do with this program, then we can perhaps give you advice that makes sense. – Jonathon Reinhart Jul 22 '12 at 04:16
Ok, I want to load up files and get there bytes, convert them by doing Convert.ToBase64String then make a textbox's text the converted bytes. But when I do this and the file is too big (about 100 megs) I get an out of memory exception. So Instead I want to load the array in chunks, convert each chunk to base 64 string and add it to the text boxes text. @Jonathon Reinhart – Layne3 Jul 22 '12 at 04:43
And where do you think that text in the text box goes? In memory. I don't think you really want to show a 100MB text file in a text box. – Jonathon Reinhart Jul 22 '12 at 04:44

score 0 · Answer 4 · answered Jul 22 '12 at 02:13

0

Depending on the file structure, it might be easier for you to use a StreamReader which exposes a ReadLine method.

using(var sr = new StreamReader(File.Open(textBox2.Text, FileMode.Open))
{
    while (sr.Peek() >= 0) 
    {
        Console.WriteLine(sr.ReadLine());
     }
}

answered Jul 22 '12 at 02:13

scottm

27,829
22
107
159

Ok, Instead of writing the line, how can I add each chunk to one array which I will convert to a string. Or convert each byte to a string and add to another string? – Layne3 Jul 22 '12 at 02:27

score 0 · Answer 5 · answered Jul 22 '12 at 07:06

If the file is a text file you can use a TextReader:

   string location1 = textBox2.Text;
    string text  = String.Empty;
    using (TextReader reader = File.OpenText(location1 ))
    {
           do
           {
           string line = reader.ReadLine();
               text+=line;
            }
            while(line!=null)

    }

How do I use File.ReadAllBytes In chunks

5 Answers5

Linked