String Truncation during Json Deserialization and ADO Persistance

Question

I am writing a console app that will hit a REST api, deserialize the JSON to a C# model and then persist that model using ADO into a db table.

First issue is that during the first run of the app we found that one of the JSON properties exceeded the column definition of nvarchar(300). We increased that column to nvarchar(4000), but I have no idea if any of the other dozen string properties might exceed the default of 300 I gave them.

FYI, the SQL error I got was:

String or binary data would be truncated.

The data for table-valued parameter "@Items" doesn't conform to the table type of the parameter. SQL Server error is: 8152, state: 10

The statement has been terminated.

...which makes sense if I was passing in a string with length 500 to an nvarchar(300)

So my desire: during deserialization or model creation in C# I would like to truncate the string properties/fields and give them a max length before I hit my persistence code so that I can ensure with 100% confidence that my fields will never exceed the nvarchar lengths and will never trigger the 'truncation error'.

I tried using System.ComponentModel.DataAnnotations and [MaxLength(4000)], but that seemed only for MVC and input validation during form POSTing.

I thought about making backing fields with custom setters, but that means having twice the lines of code in each of my entities. I have 9 entities and each probably has 2 dozen strings that I want to configure/truncate.

So question: is there any fancy way to truncate strings using some sort of NewtonSoft data annotation or a C# data annotation? Also, is there a magic way to avoid having a bazillion back fields? Or should I just make a custom string class and inherit from String that has a max length property?

Answer 1

Json.Net does not have built-in string truncation capabilities, but you could use a custom ContractResolver in combination with a custom ValueProvider to do what you want. The ContractResolver would look for string properties in all of your classes and apply the ValueProvider to them, while the ValueProvider would do the actual truncating during deserialization. You could make the resolver use a default max length of 300 (or whatever), but also look for any [MaxLength] attributes (from System.ComponentModel.DataAnnotations) you may have applied to your string properties and use that length instead as an override. So that would handle the length 4000 case.

Here is the code you would need:

public class StringTruncatingPropertyResolver : DefaultContractResolver
{
    public int DefaultMaxLength { get; private set; }

    public StringTruncatingPropertyResolver(int defaultMaxLength)
    {
        DefaultMaxLength = defaultMaxLength;
    }

    protected override IList<JsonProperty> CreateProperties(Type type, MemberSerialization memberSerialization)
    {
        IList<JsonProperty> props = base.CreateProperties(type, memberSerialization);

        // Apply a StringTruncatingValueProvider to all string properties
        foreach (JsonProperty prop in props.Where(p => p.PropertyType == typeof(string)))
        {
            var attr = prop.AttributeProvider
                           .GetAttributes(true)
                           .OfType<MaxLengthAttribute>()
                           .FirstOrDefault();
            int maxLength = attr != null ? attr.Length : DefaultMaxLength;
            prop.ValueProvider = new StringTruncatingValueProvider(prop.ValueProvider, maxLength);
        }

        return props;
    }

    class StringTruncatingValueProvider : IValueProvider
    {
        private IValueProvider InnerValueProvider { get; set; }
        private int MaxLength { get; set; }

        public StringTruncatingValueProvider(IValueProvider innerValueProvider, int maxLength)
        {
            InnerValueProvider = innerValueProvider;
            MaxLength = maxLength;
        }

        // GetValue is called by Json.Net during serialization.
        // The target parameter has the object from which to read the string;
        // the return value is a string that gets written to the JSON.
        public object GetValue(object target)
        {
            return InnerValueProvider.GetValue(target);
        }

        // SetValue gets called by Json.Net during deserialization.
        // The value parameter has the string value read from the JSON;
        // target is the object on which to set the (possibly truncated) value.
        public void SetValue(object target, object value)
        {
            string s = (string)value;
            if (s != null && s.Length > MaxLength)
            {
                s = s.Substring(0, MaxLength);
            }
            InnerValueProvider.SetValue(target, s);
        }
    }
}

To use the resolver, add it to an instance of JsonSerializerSettings and pass the settings to JsonConvert.DeserializeObject like this:

var settings = new JsonSerializerSettings
{
    ContractResolver = new StringTruncatingPropertyResolver(300)
};
var foo = JsonConvert.DeserializeObject<Foo>(json, settings);

Here is a working demo: https://dotnetfiddle.net/YOGsP5