List items changes as I add new data... why?

Question

I have a method that converts a file to bytes so that I can later send it over the internet. anyways because I plan to send large files I send chunks of files instead of sending the whole file. each chunk consist of an array of bytes (byte[]) . I am new to all this so I wanted to save each chunk in an List of chunks ( List ) before sending it . so my class looks like:

public class SomeClass
{

    public List<byte[]> binaryFileList;

    public void SendChunk(byte[] data, int index)
    {
        binaryFileList.Add(data);
        // later I will add code in here to do something with data
    }

    public void test(string path)
    {
        binaryFileList = new List<byte[]>();

        System.IO.FileStream stream = new System.IO.FileStream(path,
            System.IO.FileMode.Open, System.IO.FileAccess.Read);

        var MaxChunkSize = 10000;
        byte[] chunk = new byte[MaxChunkSize];
        while (true)
        {
            int index = 0;
            // There are various different ways of structuring this bit of code.
            // Fundamentally we're trying to keep reading in to our chunk until
            // either we reach the end of the stream, or we've read everything we need.
            while (index < chunk.Length)
            {
                int bytesRead = stream.Read(chunk, index, chunk.Length - index);

                if (bytesRead == 0)
                {
                    break;
                }
                index += bytesRead;
            }
            if (index != 0) // Our previous chunk may have been the last one
            {
                SendChunk(chunk, index); // index is the number of bytes in the chunk
            }
            if (index != chunk.Length) // We didn't read a full chunk: we're done
            {
                return;
            }
        }


    }
}

and when I execute:

SomeClass s = new SomeClass();
s.test(@"A:\Users\Tono\Desktop\t.iso");

binaryFileList List gets populated with chunks of the file: A:\Users\Tono\Desktop\t.iso

Now the problem came when I tied to create a file from that data. when debuging I noticed that the problem was because items in binaryFileList changed as I entered data. let me show you what I mean:

notice that in this debug it is the first time I add an item to binaryFileList. and also you can see each byte of that item in the array...

now I will let the method run more times adding more items to binaryFileList.

so now binaryFileList has 278 items instead of one like on the last picture:

so everything so far looks ok right? but did you guys recall that the first item of binaryFileList contained an array of bytes with almost all 0's? take a look at the first item of binaryFileList:

and as I keep adding items to binaryFileList note how the first item changes:

In other words binaryFileList is a list of byte[]. and when I add a byte[] to binaryFileList other byte[] should not change. they do change! why!?

Answer 1

The following line has to go inside the loop:

byte[] chunk = new byte[MaxChunkSize];

You create the chunk only once and overwrite it each time with new data. What you store in you list, is just a reference to this chunk, not a copy of it.

Answer 2

You are using the same reference of byte[] for chunk when you call stream.Read.

Answer 3

You are reading into the same chunk each time, and adding that same chunk with the lastest value into the list each time. To correct you need to create a new byte[] each time:

    while (true)
    {
        // *** need to create new array each time...
        var chunk = new byte[MaxChunkSize];

        int index = 0;
        // There are various different ways of structuring this bit of code.
        // Fundamentally we're trying to keep reading in to our chunk until
        // either we reach the end of the stream, or we've read everything we need.
        while (index < chunk.Length)
        {
            int bytesRead = stream.Read(chunk, index, chunk.Length - index);

            if (bytesRead == 0)
            {
                break;
            }
            index += bytesRead;
        }
        if (index != 0) // Our previous chunk may have been the last one
        {
            SendChunk(chunk, index); // index is the number of bytes in the chunk
        }
        if (index != chunk.Length) // We didn't read a full chunk: we're done
        {
            return;
        }
    }

Answer 4

My guess would be that you are re-using the byte array that you are passing to the SendChunk method. Arrays are reference types- you should create a new byte array for each method call.

Answer 5

You keep adding your buffer variable to the array, which puts the same reference to the same object in each list position.

You need to allocate a new byte[] and copy the buffer array into that new array, or alternately allocate a new buffer array for each chunk you read.

Here's an excellent two-part post on references by Eric Lippert that I suggest you read to better understand the issue:

http://blogs.msdn.com/b/ericlippert/archive/2011/03/07/references-and-pointers-part-one.aspx

http://blogs.msdn.com/b/ericlippert/archive/2011/03/10/references-and-pointers-part-two.aspx

Answer 6

The problem is because you allocated the chunk array only once:

byte[] chunk = new byte[MaxChunkSize];

and you are reading new portion of file to the same array again and again. Remember that array is passed as an reference in method parameter. Move declaration inside your loop and you should be fine.