GetBytes returning different size than data byte actually is

Question

The following code iterates through a List<byte[]>() however the problem is that for some reason, sometimes the lengthBuffer is a different size of the byteData.

You can see in the DebugPrintInGame method that I print the BitConverter.ToInt32() length and then the actual byteDataList[i].Length, these two numbers should be exactly the same and a lot of the time they are, but for reasons unknown to me a lot of the time now they aren't either.

With this code below it sends the length of the data to the server so server knows size of incoming data and what to expect, then it sends the actual data. However sometimes for example it will send the length as larger than the actual data sent but it happens randomly, which led me to believe maybe it is a threading issue - nonetheless byteDataList is only used within this one thread so its pointless for me to even lock it. In what I've tested, the data I'm sending is no more than 2kb to 17kb.

Does anyone see why this might be happening based on this little snipit from my project? This code is all client side. Notice BitConverter.ToInt32(lengthBuffer, 0) and byteDataList[i].Length is expected to give same length but it does not. That is the problem.

List<byte[]> byteDataList = new List<byte[]>();
//.........

lock (byteDataList) //pointless lock, just used for random testing :\
{
      if (byteDataList.Count > 0)
      {
             for (int i = 0; i < byteDataList.Count; i++)
             {
                 try
                 {
                      byte[] lengthBuffer = BitConverter.GetBytes(byteDataList[i].Length);
                      //quick printout to compare lengths for testing
                      this.QueueOnMainThread(() =>
                      {
                        Tools.DebugPrintInGame("Length buffer " + lengthBuffer.Length + 
                                               +  BitConverter.ToInt32(lengthBuffer,0)+ " "+
                                               + byteDataList[i].Length);  //this should be same as BitConverter lengthBuffer
                      });
                      //send length
                      stream.Write(lengthBuffer, 0, lengthBuffer.Length);
                      //send data
                      stream.Write(byteDataList[i], 0, byteDataList[i].Length);
                      stream.Flush();
                  }
                  catch (Exception ex)
                  {
                       this.QueueOnMainThread(() =>
                       {
                           Tools.DebugPrintInGame("Exception " + ex.ToString());
                       });
                  }
             }

      }
}

Print out from client side before it's even sent to server:

Notice different lengths when the last two should be the same. I.e.

 `lengthBuffer`    = 17672 
 `byteDataList[i]` = 17672 

but sometimes it goes weird and wonky:

 `lengthBuffer`    = 17672
 `byteDataList[i]` = 2126

Both lengthBuffer and byteDataList[i] should be the same.

Answer 1

This is most likely a problem with closure over the i variable in your loop (I assume you're using .NET 4.0 or earlier?) The culprit is here:

this.QueueOnMainThread(() =>
                      {
                        Tools.DebugPrintInGame("Length buffer " + lengthBuffer.Length + 
                                               +  BitConverter.ToInt32(lengthBuffer,0)+ " "+
                                               + byteDataList[i].Length);  //this should be same as BitConverter lengthBuffer
                      });

See how you reference the variable i above? The above anonymous method forms a closure over that variable, which means i is not evaluated until the method is called. This means that the value of i may no longer be the one that you used to create lengthBuffer, so you get a mismatch.

To fix this, store a copy of i into a variable in the inner scope of the loop, like this:

int k = i;
this.QueueOnMainThread(() =>
                          {
                            Tools.DebugPrintInGame("Length buffer " + lengthBuffer.Length + 
                                                   +  BitConverter.ToInt32(lengthBuffer,0)+ " "+
                                                   + byteDataList[k].Length);  //this should be same as BitConverter lengthBuffer
                          });

By closing over an inner scope variable, it will have the intended value and won't change as the loop executes. The confusion here is that a loop variable is actually considered to be out of the scope of the loop body for the purpose of closure. In later versions of C# (for .NET 4.5), they changed this behavior and the loop variable is now closed over with inner scope. They made the change because it was too easy to trip over this problem in exactly this type of situation.