Reading multiple JSON objects (Concatenated JSON) using Newtonsoft.JSON

Question

C# Newtonsoft.JSON library works very well in most situation but there is no easy way to read concatenated JSON like one below:

{"some":"thing"}{"some":"other thing"}{"some":"third thing"}

This code that pretty straightforward but it throws exception if I try to deserialize more than one object:

        using (var reader = new StreamReader(File.Open("data.txt", FileMode.Open)))
        {
            using (var jr = new JsonTextReader(reader))
            {
                var data1 = js.Deserialize<Data>(jr));
                var data2 = js.Deserialize<Data>(jr)); // <--- Exception is thrown here
            }
        }

There are couple workarounds. First one is reformatting the whole list of objects into a JSON array. Such approach works fine for small amounts of data but if file does not fit in memory then situation becomes quite complicated.

The other workaround is splitting whole text into separate JSON objects and parsing one object at a time. This solution would handle big amounts of data but implementation is a bit more complex since it requires some sort of JSON parsing.

Is there an easier way to read JSON objects in such concatenated JSON file?

Optional Option · Accepted Answer · 2019-01-02T23:09:27.987

This solution is based on Newtonsoft.JSON version 12.0.1. It may or may not work in the future.

First we need a better JsonTextReader which would reset to a usable state after deserializing JSON object.

    public class MyJsonTextReader : JsonTextReader
    {
        public MyJsonTextReader(TextReader textReader) : base(textReader)
        {
            SupportMultipleContent = true;
        }

        public bool ObjectDone()
        {
            base.SetStateBasedOnCurrent();
            try
            {
                // This call works fine at the end of the file but may throw JsonReaderException
                // if some bad character follows our JSON object
                return !base.Read();
            }
            catch (JsonReaderException)
            {
                return true;
            }
        }
    }

Using the new JSON reader class deserialization code can be slightly modified like this:

        var all = new List<Data>();
        var js = new JsonSerializer();
        using (var reader = new StreamReader(File.Open("data.txt", FileMode.Open)))
        using (var jr = new MyJsonTextReader(reader))
        do
        {
            all.Add(js.Deserialize<Data>(jr));
        } while (!jr.ObjectDone());

This solution can read unlimited number of objects. ObjectDone() function returns false at the end of the file or in case of invalid character following deserialized object.

You can reduce nesting by removing the `{` between the using statements. If one `using` statement appears one after the other, the `{}` scope is implied. See https://stackoverflow.com/questions/23197606/best-practice-for-nested-using-statements — , Dec 28 '18 at 19:11

Reading multiple JSON objects (Concatenated JSON) using Newtonsoft.JSON

1 Answers1