Is there a high performance way to replace the BinaryFormatter in .NET5?

Question

Before .NET5 we serialize/deserialize the Bytes/Object by these code:

    private static byte[] StructToBytes<T>(T t)
    {
        using (var ms = new MemoryStream())
        {
            var bf = new BinaryFormatter();
            bf.Serialize(ms, t);
            return ms.ToArray();
        }
    }

    private static T BytesToStruct<T>(byte[] bytes)
    {
        using (var memStream = new MemoryStream())
        {
            var binForm = new BinaryFormatter();
            memStream.Write(bytes, 0, bytes.Length);
            memStream.Seek(0, SeekOrigin.Begin);
            var obj = binForm.Deserialize(memStream);
            return (T)obj;
        }
    }

But the BinaryFormatter will be removed for the security reason:

https://learn.microsoft.com/en-us/dotnet/standard/serialization/binaryformatter-security-guide

So is there some simple but high performance method to replace BinaryFormatter?

Answer 1

In my project, which we recently migrated from .NET Core 3.1 to .NET 5, I swapped out our BinarySerializer code with Protobuf-net: https://github.com/protobuf-net/protobuf-net

The code was almost exactly the same, and the project is very reputable with (currently) 22 million downloads and 3.2k stars on GitHub. It is very fast and has none of the security baggage surrounding BinarySerializer.

Here's my class for byte[] serialization:

public static class Binary
{
    /// <summary>
    /// Convert an object to a Byte Array, using Protobuf.
    /// </summary>
    public static byte[] ObjectToByteArray(object obj)
    {
        if (obj == null)
            return null;

        using var stream = new MemoryStream();

        Serializer.Serialize(stream, obj);

        return stream.ToArray();
    }

    /// <summary>
    /// Convert a byte array to an Object of T, using Protobuf.
    /// </summary>
    public static T ByteArrayToObject<T>(byte[] arrBytes)
    {
        using var stream = new MemoryStream();

        // Ensure that our stream is at the beginning.
        stream.Write(arrBytes, 0, arrBytes.Length);
        stream.Seek(0, SeekOrigin.Begin);

        return Serializer.Deserialize<T>(stream);
    }
}

I did have to add attributes to the class I serialized. It was decorated with [Serializable] only, and although I understand Protobuf can work with a lot of common decorations, that one didn't work. From the example on github:

[ProtoContract]
class Person {
    [ProtoMember(1)]
    public int Id {get;set;}
    [ProtoMember(2)]
    public string Name {get;set;}
    [ProtoMember(3)]
    public Address Address {get;set;}
}

[ProtoContract]
class Address {
    [ProtoMember(1)]
    public string Line1 {get;set;}
    [ProtoMember(2)]
    public string Line2 {get;set;}
}

In my case I am caching things in Redis, and it worked great.

It is also possible to re-enable this, in your .csproject file:

<PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization>
</PropertyGroup>

...But it's a bad idea. BinaryFormatter is responsible for many of .NET's historical vulnerabilities, and it can't be fixed. It will likely become completely unavailable in future versions of .NET, so replacing it is the right move.

Answer 2

if you are using .NET Core 5 or greater, you can use the new System.Text.Json.JsonSerializer.Serialize and System.Text.Json.JsonSerializer.Deserialize like so:

public static class Binary
{
    /// <summary>
    /// Convert an object to a Byte Array.
    /// </summary>
    public static byte[] ObjectToByteArray(object objData)
    {
        if (objData == null)
            return default;

       return Encoding.UTF8.GetBytes(JsonSerializer.Serialize(objData, GetJsonSerializerOptions()));
    }

    /// <summary>
    /// Convert a byte array to an Object of T.
    /// </summary>
    public static T ByteArrayToObject<T>(byte[] byteArray)
    {
        if (byteArray == null || !byteArray.Any())          
            return default;
            
        return JsonSerializer.Deserialize<T>(byteArray, GetJsonSerializerOptions());
    }

    private static JsonSerializerOptions GetJsonSerializerOptions()
    {
        return new JsonSerializerOptions()
        {
            PropertyNamingPolicy = null,
            WriteIndented = true,
            AllowTrailingCommas = true,
            DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull,
        };
    }
}

Answer 3

While an old thread, it's still relevant, especially if you find yourself dealing with code storing .NET data in Memcached for example (or Redis, or secondary storage on-prem or in a cloud). BinaryFormatter has the security problems mentioned in the OP, and also has performance and size issues.

A great alternative is the MessagePack format, and more specifically the MessagePack NuGet package for .NET solutions.

It's secure, maintained, faster, and smaller all around. See the benchmarks for details.

ZeroFormatter also appears to be a great alternative.

In today's cloud-centric solutions where sizing and capacity are important for lowering costs, these are extremely helpful.

Answer 4

There is an option to use it in .NET Core 5:

Just add

<EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization>

To the project like:

<PropertyGroup>
    <TargetFramework>net5.0</TargetFramework>
    <EnableUnsafeBinaryFormatterSerialization>true</EnableUnsafeBinaryFormatterSerialization>
</PropertyGroup>

I believe it will work.

Answer 5

There is a new System.Formats.Cbor namespace for reading and writing Concise Binary Object Representation, which is a successor to MessagePack. You need to install a NuGet package in order to use it.

Answer 6

public static T Clone<T>(this T source) where T : class
        {
            if (!typeof(T).IsSerializable)
                throw new ArgumentException(string.Format("The type '{0}' must be serializable", source.GetType().Name));

            // Don't serialize a null object, simply return the default for that object
            if (ReferenceEquals(source, null))
                return default(T);

            using (Stream stream = new MemoryStream())
            {
                var writer = new BinaryWriter(stream, Encoding.UTF8, false);
                
                DataContractJsonSerializer js = new DataContractJsonSerializer(typeof(T));
                js.WriteObject(stream, source);
                stream.Seek(0, SeekOrigin.Begin);

                // Return deserialzed object
                return (T)js.ReadObject(stream);
            }
        }