Deserializing a byte array

Question

If I wanted to fill a structure from a binary file, I would use something like this:

using (BinaryReader br = new BinaryReader(File.Open(filename, FileMode.Open)))
{
    myStruct.ID = br.ReadSingle();
    myStruct.name = br.ReadBytes(20);
}

However, I must read the whole file into a byte array before deserializing, because I want to do some pre-processing. Is there any managed way to fill my structure from the byte array, preferably similar to the one above?

You should consider making your type serializable. If that is something you are interested in, i will provide a sample. See "BinaryFormatter" for Binary Serialization. — Glenn Ferrie, Jul 05 '11 at 18:10
@Nate, thanks, it seems using `MemoryStream` is a good idea! @GlennFerrieLive, I've never worked with `BinaryFormatter` before, but judging by some examples it looks like "casting to a struct" for me. I'll really appreciate even a small sample. Thanks! — skink, Jul 05 '11 at 18:18
In my experience `BinaryFormatter` is rarely the correct choice for serializing data. — CodesInChaos, Jul 05 '11 at 19:08
@CodeInChaos, what's the correct one by your opinion? Thanks! — skink, Jul 05 '11 at 19:10
That depends very much on the situation. I usually need a well defined file format, so I use something like Linq-to-Json or Linq-to-Xml to transform between my in-memory representation and the file format. Sometimes protobuf is nice because it's very compact. And in some rare cases, if you don't need versioning and can live with its deeply invasive nature `BinaryFormatter` can be the right choice. A savegame in a game is one of the few cases that fits `BinaryFormatter` IMO. — CodesInChaos, Jul 05 '11 at 19:24
Another big issue about the `BinaryFormatter` is that you must trust the file absolutely. Whoever created the file is most likely able to execute code in the context of your program. — CodesInChaos, Jul 05 '11 at 19:47
@CodeInChaos, thank you very much, I'll read about the ways of serialization you prefer! — skink, Jul 06 '11 at 12:04
Linq-To-Json/Xml technically isn't serialization. The transform between the representations is done manually, which is obviously much more work. But I think for stable file formats the clean separation of in memory representation and storage format is usually worth the additional work. — CodesInChaos, Jul 06 '11 at 12:12

score 16 · Accepted Answer · answered Jul 05 '11 at 18:24

16

This is a sample to take some data (actually a System.Data.DataSet) and serialize to an array of bytes, while compressing using DeflateStream.

try
{
    var formatter = new BinaryFormatter();
    byte[] content;
    using (var ms = new MemoryStream())
    {
         using (var ds = new DeflateStream(ms, CompressionMode.Compress, true))
         {
             formatter.Serialize(ds, set);
         }
         ms.Position = 0;
         content = ms.GetBuffer();
         contentAsString = BytesToString(content);
     }
}
catch (Exception ex) { /* handle exception omitted */ }

Here is the code in reverse to deserialize:

        var set = new DataSet();
        try
        {
            var content = StringToBytes(s);
            var formatter = new BinaryFormatter();
            using (var ms = new MemoryStream(content))
            {
                using (var ds = new DeflateStream(ms, CompressionMode.Decompress, true))
                {
                    set = (DataSet)formatter.Deserialize(ds);                        
                }
            }
        }
        catch (Exception ex)
        {
            // removed error handling logic!
        }

Hope this helps. As Nate implied, we are using MemoryStream here.

answered Jul 05 '11 at 18:24

Glenn Ferrie

10,290
3
42
73

Thanks a lot, this should be more helpful with the big structures. Just a question - does changing the structure alignments affect the deserialization result? – skink Jul 05 '11 at 18:31
I think the structure alignment affects serialization and deserialization, but I'm not certain. – Glenn Ferrie Jul 05 '11 at 18:33
1

@Angel for the record, I strongly advise ***against*** BinaryFormatter for *any* kind of storage. It has several nasty quirks, and I have honestly lost count of the number of times I've had to help people struggling with lost data etc. Other serializers exist that are (IMO, based on extensive work in serialization) more robust, just a convenient, and much more efficient for both CPU and bandwidth. – Marc Gravell Jul 05 '11 at 19:23
Hey Mark - I am using this sample in code that I am working on now, I was using XmlSerializer, but switched to BinaryFormatter. Should I go back? What are some of the other options? – Glenn Ferrie Jul 05 '11 at 19:26
@Glenn That depends on what you want to use the serializer for. – CodesInChaos Jul 05 '11 at 19:46

score 1 · Answer 2 · answered Jul 05 '11 at 18:17

1

Take a look at the BitConverter class. That might do what you need.

answered Jul 05 '11 at 18:17

Joel Coehoorn

399,467
113
570
794

Thanks for the answer, that class really does what I need. The only small inconvenience is that I need to track the position manually. – skink Jul 05 '11 at 18:24

score 0 · Answer 3 · answered Nov 22 '14 at 21:48

For very simple structs which aren't Serializable and contain only base types, this works. I use it for parsing files which have a known format. Error checking removed for clarity.

using System;
using System.IO;
using System.Reflection;
using System.Runtime.InteropServices;

namespace FontUtil
{
    public static class Reader
    {
        public static T Read<T>(BinaryReader reader, bool fileIsLittleEndian = false)
        {
            Type type = typeof(T);
            int size = Marshal.SizeOf(type);
            byte[] buffer = new byte[size];
            reader.Read(buffer, 0, size);
            if (BitConverter.IsLittleEndian != fileIsLittleEndian)
            {
                FieldInfo[] fields = type.GetFields();
                foreach (FieldInfo field in fields)
                {
                    int offset = (int)Marshal.OffsetOf(type, field.Name);
                    int fieldSize = Marshal.SizeOf(field.FieldType);
                    for (int b = offset, t = fieldSize + b - 1; b < t; ++b, --t)
                    {
                        byte temp = buffer[t];
                        buffer[t] = buffer[b];
                        buffer[b] = temp;
                    }
                }
            }
            GCHandle h = GCHandle.Alloc(buffer, GCHandleType.Pinned);
            T obj = (T)Marshal.PtrToStructure(h.AddrOfPinnedObject(), type);
            h.Free();
            return obj;
        }
    }
}

Structs need to be declared like this (and can't contain arrays, I think, haven't tried that out - the endian swap would probably get confused).

[StructLayout(LayoutKind.Sequential, Pack = 1)]
public struct NameRecord
{
    public UInt16 uPlatformID;
    public UInt16 uEncodingID;
    public UInt16 uLanguageID;
    public UInt16 uNameID;
    public UInt16 uStringLength;
    public UInt16 uStringOffset; //from start of storage area
}

Deserializing a byte array

3 Answers3