oneway IEnumerable processing

Question

I have huge source (some IEnumerable that goes through an IDbReader row by row), that I need to serialize to yaml and deserialize back.
How can I avoid collecting all items in memory?

Answer 1

You should be able to use the Serializer directly to serialize the IEnumerable. Be sure to disable aliases on the serializer and it should serialize in a streaming fashion, without loading the entire source first:

var serializer = new SerializerBuilder()
    .DisableAliases()
    .Build();

You can see this in action in the following code. It will serialize the first 100 items then fail with an exception, but you can see that the first items were already serialized:

using System;
using YamlDotNet.Serialization;
using System.Collections.Generic;
using System.Linq;

public class Program
{
    public static void Main()
    {
        var source = Enumerable.Range(1, 10000).Select(i => {

            if(i == 100) throw new Exception("I'm done");

            return new {
                Index = i,
                Title = "Item " + i
            };
        });

        var serializer = new SerializerBuilder()
            .DisableAliases()
            .Build();

        serializer.Serialize(Console.Out, source);
    }
}

See it running here: https://dotnetfiddle.net/Rk1nrx