How to serialize/deserialize json dictionary into array

Question

Need to serialize and deserialize C# dictionaries into a JSON array. I would like to also read the JSON from powershell using array index notation.

By default, JSON format is:

{
 "defaultSettings": {
  "applications": {
   "Apollo": {
    "environments": {
      "DEV": {
        "dbKeyTypes": {
          "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06",
          "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
        }
      },
      "TST": {
        "dbKeyTypes": {
          "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06",
          "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
        }
      }
    }
  },
  "Gemini": {
    "environments": {
      "DEV": {
        "dbKeyTypes": {
          "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06",
          "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
        }
      },
      "TST": {
        "dbKeyTypes": {
          "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06",
          "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
        }
      }
    }
   }
  }
 }
}

This works great using the default json reader in .Net Core, but it doesn't allow me to use array index notation in PowerShell.

Instead what I'm looking for is this:

{
 "defaultSettings": {
  "applications": [
   {
     "Apollo": {
      "environments": [
        {
          "DEV": {
            "dbKeyTypes": [
              {
                "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06"
              },
              {
                "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
              }
            ]
          }
        },
        {
          "TST": {
            "dbKeyTypes": [
              {
                "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06"
              },
              {
                "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
              }
            ]
          }
        }
      ]
    }
  },
  {
    "Gemini": {
      "environments": [
        {
          "DEV": {
            "dbKeyTypes": [
              {
                "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06"
              },
              {
                "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
              }
            ]
          }
        },
        {
          "TST": {
            "dbKeyTypes": [
              {
                "DmkPassword": "AEikOooIuGxXC9UBJQ3ckDj7Q126tB06"
              },
              {
                "SymmetricKeySource": "bTU7XOAYA2FFifmiBUggu99yHxX3Ftds"
              }
            ]
          }
        }
      ]
    }
   }
  ]
 }
}

I'm using the WriteJson part from Serializing Dictionary<string,string> to array of "name": "value"

This works well; however, of course since the ReadJson() method isn't implemented, it doesn't read. Btw, to get the above desired json format, I modified the CustomDictionaryConverter in the link to:

writer.WritePropertyName(key.ToString());
//writer.WriteValue(key);
//writer.WritePropertyName("value");
serializer.Serialize(writer, valueEnumerator.Current);

The classes behind the implementation are:

public enum DeploymentEnvironment { DEV = 1, TST = 2 }
public enum TargetApplication { Apollo = 1, Gemini = 2 }
public enum DbKeyType { DmkPassword = 1, SymmetricKeySource = 2 }

public class DeploymentSettings
{
    [JsonProperty("defaultSettings")]
    public DefaultSettings DefaultSettings { get; set; }
    public DeploymentSettings()
    {
        DefaultSettings = new DefaultSettings();
    }
}

public partial class DefaultSettings
{
    [JsonProperty("applications")]
    public Dictionary<TargetApplication, ApplicationContainer> Applications { get; set; }

    public DefaultSettings()
    {
        Applications = new Dictionary<TargetApplication, ApplicationContainer>();
    }
}

public partial class ApplicationContainer
{
    [JsonProperty("environments")]
    public Dictionary<DeploymentEnvironment, EnvironmentContainer> Environments { get; set; }
    public ApplicationContainer()
    {
        Environments = new Dictionary<DeploymentEnvironment, EnvironmentContainer>();
    }
}

public partial class EnvironmentContainer
{
    [JsonProperty("dbKeyTypes")]
    public Dictionary<DbKeyType, string> DbKeyTypes { get; set; }

    public EnvironmentContainer()
    {
        DbKeyTypes = new Dictionary<DbKeyType, string>();
    }
}

I'm serializing the object as follows: var json = JsonConvert.SerializeObject(ds, Formatting.Indented, new CustomDictionaryConverter());

As mentioned, serializing works, but I need help writing the ReadJson() method in order to be able to deserialize.

Ideal way of posting this problem is not such a long and complex Json, ideally you shall post a sub part, get the solution and use to it for the complex version. In any case you don't need both Write and Read, if you have already written a Json, then simply read back in the correct schema, no read required. In this case you just need concatenated Dictionary Write to take care of all the nested objects — Mrinal Kamboj, May 20 '19 at 05:50
Thanks for posting (and that was the condensed version as the real json format is much more complex). I do need a read and a write because I'm converting data from a different format by programmatically populating the listed classes and then serializing them. Then json gets modified in an outside operation which I will read later by deserializing it into the classes. I am looking for help (i.e. code snippet) to be able to do that. — Arjay H., May 20 '19 at 13:31

dbc · Accepted Answer · 2019-05-20T23:32:14.837

You can extend CustomDictionaryConverter to read as well as write as follows:

public class CustomDictionaryConverter : JsonConverter
{
    // Adapted from CustomDictionaryConverter from this answer https://stackoverflow.com/a/40265708
    // To https://stackoverflow.com/questions/40257262/serializing-dictionarystring-string-to-array-of-name-value
    // By Brian Rogers https://stackoverflow.com/users/10263/brian-rogers

    sealed class InconvertibleDictionary : Dictionary<object, object>
    {
        public InconvertibleDictionary(DictionaryEntry entry)
            : base(1)
        {
            this[entry.Key] = entry.Value;
        }
    }

    public override bool CanConvert(Type objectType)
    {
        return typeof(IDictionary).IsAssignableFrom(objectType) && objectType != typeof(InconvertibleDictionary);
    }

    public override void WriteJson(JsonWriter writer, object value, JsonSerializer serializer)
    {
        // Lazy evaluation of the enumerable prevents materialization of the entire collection of dictionaries at once.
        serializer.Serialize(writer,  Entries(((IDictionary)value)).Select(p => new InconvertibleDictionary(p)));
    }

    public override object ReadJson(JsonReader reader, Type objectType, object existingValue, JsonSerializer serializer)
    {
        if (reader.MoveToContentAndAssert().TokenType == JsonToken.Null)
            return null;
        var dictionary = existingValue ?? serializer.ContractResolver.ResolveContract(objectType).DefaultCreator();
        switch (reader.TokenType)
        {
            case JsonToken.StartObject:
                serializer.Populate(reader, dictionary);
                return dictionary;

            case JsonToken.StartArray:
                {
                    while (true)
                    {
                        switch (reader.ReadToContentAndAssert().TokenType)
                        {
                            case JsonToken.EndArray:
                                return dictionary;

                            case JsonToken.StartObject:
                                serializer.Populate(reader, dictionary);
                                break;

                            default:
                                throw new JsonSerializationException(string.Format("Unexpected token {0}", reader.TokenType));
                        }
                    }
                }

            default:
                throw new JsonSerializationException(string.Format("Unexpected token {0}", reader.TokenType));
        }
    }

    static IEnumerable<DictionaryEntry> Entries(IDictionary dict)
    {
        foreach (DictionaryEntry entry in dict)
            yield return entry;
    }
}

public static partial class JsonExtensions
{
    public static JsonReader ReadToContentAndAssert(this JsonReader reader)
    {
        return reader.ReadAndAssert().MoveToContentAndAssert();
    }

    public static JsonReader MoveToContentAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (reader.TokenType == JsonToken.None)       // Skip past beginning of stream.
            reader.ReadAndAssert();
        while (reader.TokenType == JsonToken.Comment) // Skip past comments.
            reader.ReadAndAssert();
        return reader;
    }

    public static JsonReader ReadAndAssert(this JsonReader reader)
    {
        if (reader == null)
            throw new ArgumentNullException();
        if (!reader.Read())
            throw new JsonReaderException("Unexpected end of JSON stream.");
        return reader;
    }
}

You can then serialize and deserialize your DeploymentSettings with the following settings:

var settings = new JsonSerializerSettings
{
    Converters = { new CustomDictionaryConverter(), new StringEnumConverter() }
};

var ds = JsonConvert.DeserializeObject<DeploymentSettings>(json, settings);

var json2 = JsonConvert.SerializeObject(ds, Formatting.Indented, settings);

Notes:

This version of the converter avoids loading the entire dictionary into a temporary JArray hierarchy in either ReadJson() or WriteJson(), and instead streams directly from and to the JSON stream.
Because the serializer is now used to directly serialize the individual dictionary entries, StringEnumConverter is required for the keys to be named correctly. (Using the serializer also ensures that numeric or DateTime keys are properly internationalized, should you be using such a dictionary anywhere.)
Because Json.NET supports comments, the converter checks for and skips them, which adds a bit to the complexity. (I wish there were a way to make JsonReader silently skip over comments.)

Demo fiddle here.

Say, how long has the editing of posts been going on at stackoverflow? I've received one extremely useful answer and several useless edits. Even the spot on answer got edited. What gives? — Arjay H., May 21 '19 at 07:24
You can see the revision history showing all changes by clicking on the `edited X hours ago` link, e.g. https://stackoverflow.com/posts/56229345/revisions. I clarified a sentence in one of the descriptive paragraphs. — dbc, May 21 '19 at 07:30
Ah, I see. You edited your own work, then. Perfectly good (and thanks again). I was finding the wordsmith editing by non-technical 'contributors' to be mildly annoying. — Arjay H., May 21 '19 at 11:07

How to serialize/deserialize json dictionary into array

1 Answers1