PATCH entity and related entities in Entity Framework Core via .NET Core Web API

Question

I have a Web API written in .NET Core which uses EF Core to manage inserts and queries to a postgresql database. The API works great for inserts and queries of existing entities, but I am having trouble working out how to do partial 'patch' updates. The client wants to be able to pass only the attributes they wish to update. So a full customer JSON payload may look like this:

{
    "customer": {
        "identification": {
            "membership_number": "2701138910268@priceline.com.au",
            "loyalty_db_id": "4638092"
        },
        "name": {
            "title": "Ms",
            "first_name": "tx2bxtqoa",
            "surname": "oe6qoto"
        },
        "date_of_birth": "1980-12-24T00:00:00",
        "gender": "F",
        "customer_type": "3",
        "home_store_id": "777",
        "home_store_updated": "1980-12-24T00:00:00",
        "store_joined_id": "274",
        "store_joined_date": "1980-12-24T00:00:00",
        "status_reason": null,
        "status": "50",
        "contact_information": [
            {
                "contact_type": "EMAIL",
                "contact_value": "2yupelxqui@hotmails.com",
                "validated": true,
                "updating_store": null
            },
            {
                "contact_type": "MOBILE",
                "contact_value": "xxxxxxxxx",
                "validated": true,
                "updating_store": null
            }
        ],
        "marketing_preferences": [],
        "address": {
            "address_line_1": "something stree",
            "address_line_2": "Snyder",
            "postcode": "3030"
        },
        "external_cards": [
            {
                "updating_store": null,
                "card_type": "PY",
                "card_design": null,
                "card_number": "2701138910268",
                "status": "ACTIVE",
                "physical_print": false
            }
        ]
    }
}

But the client wants to pass in a payload like:

{
    "customer": {
        "identification": {
            "membership_number": "2701138910268@priceline.com.au"
        },
        "address": {
            "address_line_1": "something stree"
        },
    }
}

And have only the address_line_1 property updated. The rest of the fields are to remain as is. Unfortunately, because I convert the JSON into a CustomerPayload object, then the CustomerPayload object into a Customer object (and related entities), if a property is not passed, then it is set to NULL.

This means when I use SetValues in EF Core to copy properties across, those not provided are set to NULL, then updated in the database as NULL. Short of asking the client to pass all properties through and just pass existing values for the properties to be left unchanged, I am unsure how to deal with this.

So once the incoming JSON is converted to CustomerPayload (and attributes are validated) I use the below to convert CustomerPayload to Customer:

public Customer Convert(CustomerPayload source)
{
    Customer customer = new Customer
            {
                McaId = source.RequestCustomer.Identification.MembershipNumber,
                BusinessPartnerId = source.RequestCustomer.Identification.BusinessPartnerId,
                Status = source.RequestCustomer.Status,
                StatusReason = source.RequestCustomer.StatusReason, 
                LoyaltyDbId = source.RequestCustomer.Identification.LoyaltyDbId,
                Gender = source.RequestCustomer.Gender,
                DateOfBirth = source.RequestCustomer.DateOfBirth,
                CustomerType = source.RequestCustomer.CustomerType,
                HomeStoreId = source.RequestCustomer.HomeStoreId,
                HomeStoreUpdated = source.RequestCustomer.HomeStoreUpdated,
                StoreJoined = source.RequestCustomer.StoreJoinedId,
                CreatedDate = Functions.GenerateDateTimeByLocale(),
                UpdatedBy = Functions.DbUser
            };

    if (source.RequestCustomer.Name != null)
    {
        customer.Title = source.RequestCustomer.Name.Title;
        customer.FirstName = source.RequestCustomer.Name.FirstName;
        customer.LastName = source.RequestCustomer.Name.Surname;
    }

    if (source.RequestCustomer.Address != null)
    {
        customer.Address.Add(new Address
                {
                    AddressType = source.RequestCustomer.Address.AddressType,
                    AddressLine1 = source.RequestCustomer.Address.AddressLine1,
                    AddressLine2 = source.RequestCustomer.Address.AddressLine2,
                    Suburb = source.RequestCustomer.Address.Suburb,
                    Postcode = source.RequestCustomer.Address.Postcode,
                    Region = source.RequestCustomer.Address.State, 
                    Country = source.RequestCustomer.Address.Country,
                    CreatedDate = Functions.GenerateDateTimeByLocale(),
                    UpdatedBy = Functions.DbUser,
                    UpdatingStore = source.RequestCustomer.Address.UpdatingStore,
                    AddressValidated = source.RequestCustomer.Address.AddressValidated,
                    AddressUndeliverable = source.RequestCustomer.Address.AddressUndeliverable
                });
    }

    if (source.RequestCustomer.MarketingPreferences != null)
    {
        customer.MarketingPreferences = source.RequestCustomer.MarketingPreferences
                    .Select(x => new MarketingPreferences()
                    {
                        ChannelId = x.Channel,
                        OptIn = x.OptIn,
                        ValidFromDate = x.ValidFromDate,
                        UpdatedBy = Functions.DbUser,
                        CreatedDate = Functions.GenerateDateTimeByLocale(),
                        UpdatingStore = x.UpdatingStore,
                        ContentTypePreferences = (from c in x.ContentTypePreferences
                            where x.ContentTypePreferences != null
                            select new ContentTypePreferences
                            {
                                TypeId = c.Type,
                                OptIn = c.OptIn,
                                ValidFromDate = c.ValidFromDate,
                                ChannelId = x.Channel //TODO: Check if this will just naturally be passed in JSON so can use c. instead of x.)
                            }).ToList(),
                    })
                    .ToList();
    }

    if (source.RequestCustomer.ContactInformation != null)
    {
        // Validate email if present
        var emails = (from e in source.RequestCustomer.ContactInformation
                      where e.ContactType.ToUpper() == ContactInformation.ContactTypes.Email && e.ContactValue != null
                    select e.ContactValue);

        foreach (var email in emails)
        {
            Console.WriteLine($"Validating email {email}");

            if (!IsValidEmail(email))
            {
                throw new Exception($"Email address {email} is not valid.");
            }
        }

        customer.ContactInformation = source.RequestCustomer.ContactInformation
                    .Select(x => new ContactInformation()
                    {
                        ContactType = x.ContactType,
                        ContactValue = x.ContactValue,
                        CreatedDate = Functions.GenerateDateTimeByLocale(),
                        UpdatedBy = Functions.DbUser,
                        Validated = x.Validated,
                        UpdatingStore = x.UpdatingStore

                    })
                    .ToList();
        }

        if (source.RequestCustomer.ExternalCards != null)
        {
            customer.ExternalCards = source.RequestCustomer.ExternalCards
                    .Select(x => new ExternalCards()
                    {
                        CardNumber = x.CardNumber,
                        CardStatus = x.Status.ToUpper(),
                        CardDesign = x.CardDesign,
                        CardType = x.CardType,
                        UpdatingStore = x.UpdatingStore,
                        UpdatedBy = Functions.DbUser
                    })
                    .ToList();
        }

        Console.WriteLine($"{customer.ToJson()}");
        return customer; 
   }

Then I use the below method to update. The best compromise I have right now, is that they can omit certain sections (like Address, or anything inside Contact_information etc) and nothing will be updated, but they want full flexibility to pass individual properties, and I want to provide it. How can I restructure this so that if they don't pass specific properties for the Customer or related entities (Address etc) they are simply ignored in the SetValues or update statement generated by EF Core?

public static CustomerPayload UpdateCustomerRecord(CustomerPayload customerPayload)
    {
        try
        {
            var updateCustomer = customerPayload.Convert(customerPayload);
            var customer = GetCustomerByCardNumber(updateCustomer.ExternalCards.First().CardNumber);

            Console.WriteLine($"Existing customer {customer.McaId} will be updated from incoming customer {updateCustomer.McaId}");

            using (var loyalty = new loyaltyContext())
            {
                loyalty.Attach(customer);
               
                // If any address is provided
                if (updateCustomer.Address.Any())
                {
                    Console.WriteLine($"Update customer has an address");
                    foreach (Address a in updateCustomer.Address)
                    {
                        Console.WriteLine($"Address of type {a.AddressType}");
                        if (customer.Address.Any(x => x.AddressType == a.AddressType))
                        {
                            Console.WriteLine($"Customer already has an address of this type, so it is updated.");
                            a.AddressInternalId = customer.Address.First(x => x.AddressType == a.AddressType).AddressInternalId;
                            a.CustomerInternalId = customer.Address.First(x => x.AddressType == a.AddressType).CustomerInternalId;
                            a.CreatedDate = customer.Address.First(x => x.AddressType == a.AddressType).CreatedDate;
                            a.UpdatedDate = Functions.GenerateDateTimeByLocale();
                            a.UpdatedBy = Functions.DbUser;
                            loyalty.Entry(customer.Address.First(x => x.AddressType == a.AddressType)).CurrentValues.SetValues(a);
                        }
                        else
                        {
                            Console.WriteLine($"Customer does not have an address of this type, so it is inserted.");
                            customer.AddAddressToCustomer(a);
                        }
                    }
                }
                // We want to update contact information 
                if (updateCustomer.ContactInformation.Any())
                {
                    Console.WriteLine($"Some contact information has been provided to update");
                    foreach (var c in updateCustomer.ContactInformation)
                    {
                        Console.WriteLine($"Assessing contact information {c.ContactValue} of type {c.ContactType}");
                        if (customer.ContactInformation.Any(ci => ci.ContactType == c.ContactType))
                        {
                            Console.WriteLine($"The customer already has a contact type of {c.ContactType}");
                            // we have an existing contact of this type so update
                            var existContact = (from cn in customer.ContactInformation
                                                where cn.ContactType == c.ContactType
                                                select cn).Single();

                            Console.WriteLine($"Existing contact id is {existContact.ContactInternalId} with value {existContact.ContactValue} from customer id {existContact.CustomerInternalId} which should match db customer {customer.CustomerInternalId}");
                            // Link the incoming contact to the existing contact by Id 
                            c.CustomerInternalId = existContact.CustomerInternalId;
                            c.ContactInternalId = existContact.ContactInternalId;

                            // Set the update date time to now
                            c.UpdatedDate = Functions.GenerateDateTimeByLocale();
                            c.UpdatedBy = Functions.DbUser;
                            c.CreatedDate = existContact.CreatedDate;
                            loyalty.Entry(existContact).CurrentValues.SetValues(c);
                        }
                        else
                        {
                            Console.WriteLine($"There is no existing type of {c.ContactType} so creating a new entry");
                            // we have no existing contact of this type so create
                            customer.AddContactInformationToCustomer(c);
                        }
                    }
                }

                updateCustomer.CustomerInternalId = customer.CustomerInternalId;
                updateCustomer.CreatedDate = customer.CreatedDate;
                updateCustomer.UpdatedDate = Functions.GenerateDateTimeByLocale();

                loyalty.Entry(customer).CurrentValues.SetValues(updateCustomer);
                loyalty.Entry(customer).State = EntityState.Modified;

                if (updateCustomer.BusinessPartnerId == null)
                {
                    Console.WriteLine($"BPID not specified or NULL. Do not update.");
                    loyalty.Entry(customer).Property(x => x.BusinessPartnerId).IsModified = false;
                }

                // CustomerPayload used to check name, as Customer has no outer references/element for name details. 
                if (customerPayload.RequestCustomer.Name == null)
                {
                    loyalty.Entry(customer).Property(x => x.FirstName).IsModified = false;
                    loyalty.Entry(customer).Property(x => x.LastName).IsModified = false;
                    loyalty.Entry(customer).Property(x => x.Title).IsModified = false;
                }

                loyalty.SaveChanges();
                customerPayload = customer.Convert(customer);

                // Return customer so we can access mcaid, bpid etc. 
                return customerPayload; 
            }
        }
        catch (ArgumentNullException e)
        {
            Console.WriteLine(e);
            throw new CustomerNotFoundException();
        }
        catch (Exception ex)
        {
            Console.WriteLine($"{ex}");
            throw ex; 
        }
    }

Example of mapping Identification section:

public class Identification
{
    [DisplayName("business_partner_id")]
    [Description("A business_partner_id is required")]
    [StringLength(10)]
    [DataType(DataType.Text)]
    [JsonProperty("business_partner_id", Required = Required.Default)]
    public string BusinessPartnerId { get; set; } 

    [DisplayName("membership_number")]
    [Description("A membership_number is required")]
    [StringLength(50)]
    [DataType(DataType.Text)]
    [JsonProperty("membership_number", Required = Required.Default)]
    public string MembershipNumber { get; set; }

    [DisplayName("loyalty_db_id")]
    [Description("A loyalty_db_id is required")]
    [StringLength(50)]
    [DataType(DataType.Text)]
    [JsonProperty("loyalty_db_id", Required = Required.Default)]
    public string LoyaltyDbId { get; set; }
}

Answer 1

Okay, so i am sure i am missing something as this is absolutely bare-bones, but the basic idea is follows.

Given your DTO classes that look something like this:

    public class CustomerPayload
    {
        public Identification Identification { get; set; }

        [JsonProperty("contact_information")]
        public ContactInfo[] ContactInformation { get; set; }
    }

    public class ContactInfo
    {
        public bool Validated { get; set; }
    }

    public class Identification
    {
        [JsonProperty("membership_number")]
        public string MembershipNumber { get; set; }

        public string SomePropertyNotInPayload { get; set; }
    }

We need to declare one crutches thingy (coz for some reason your sample has a top level 'customer' property, looks like this:

    public class PartialCustomerPayloadWrapper
    {
        public JObject Customer { get; set; }
    }

Then we can have a method that does all the voodoo:

    private void SetThings(object target, JObject jObj)
    {
        var properties = target.GetType()
            .GetProperties(BindingFlags.Public | BindingFlags.Instance)
            .Select(x =>
        {
            var attr = x
            .GetCustomAttributes(typeof(JsonPropertyAttribute), false)
            .FirstOrDefault();

            string jPropName = null;
            if (attr != null)
            {
                jPropName = ((JsonPropertyAttribute)attr).PropertyName;
            }

            return (Property: x, Name: x.Name, JsonName: jPropName);
        });

        foreach (var val in jObj)
        {
            var key = val.Key.ToLowerInvariant();
            var property = properties
                .FirstOrDefault(x => x.Name.ToLowerInvariant() == key ||
                x.JsonName?.ToLowerInvariant() == key);

            if (property == default)
            {
                continue;
            }

            if (val.Value.Type == JTokenType.Object)
            {
                var newTarget = property.Property.GetValue(target);
                if (newTarget == null)
                {
                    newTarget = Activator.CreateInstance(property.Property.PropertyType);
                    property.Property.SetValue(target, newTarget);
                }

                SetThings(property.Property.GetValue(target), (JObject)val.Value);
            }
            else
            {
                property.Property.SetValue(target, val.Value.ToObject(property.Property.PropertyType));
            }
        }
    }

And finally our API action:

    [HttpPost]
    public string Post([FromBody] PartialCustomerPayloadWrapper wrapper)
    {
    // So  here i expect you to get data from DB 
    // and then pass through the method that converts the db object to `CustomerPayload`
    // Since i do not have that, this is just a POCO with some properties initialized.
        var dbCustomer = new CustomerPayload { Identification = new Identification { SomePropertyNotInPayload = "banana" } };

        var customer = wrapper.Customer;

        SetThings(dbCustomer, customer);
     // at this point our SomePropertyNotInPayload is still banana, but contact info and MembershipNumber are set
        return "OK";
    }

I used this payload for testing:

{
    "customer": {
        "identification": {
            "membership_number": "2701138910268@priceline.com.au"
        },
        "address": {
            "address_line_1": "something stree"
        },
        "contact_information": [
            {
                "contact_type": "EMAIL",
                "contact_value": "2yupelxqui@hotmails.com",
                "validated": true,
                "updating_store": null
            },
            {
                "contact_type": "MOBILE",
                "contact_value": "xxxxxxxxx",
                "validated": false,
                "updating_store": null
            }
        ]
    }
}

Note: The biggest downfall of this approach is that you can't really marry up the 'contact_info' because you need some kind of primary key (which i am assuming is already in the route for your customer). If you had that, you can extend the voodoo part by checking for JTokenType.Array and then processing individual items through the similar set up.