2

I have a DTO with a property of type Dictionary<string, string>. It's not annotated. When I upload my DTO and call indexClient.Documents.Index(batch), I get this error back from the service:

The request is invalid. Details: parameters : A node of type 'StartObject' was read from the JSON reader when trying to read the contents of the property 'Data'; however, a 'StartArray' node was expected.

The only way I've found to avoid it is by setting it to null. This is how I created my index:

var fields = FieldBuilder.BuildForType<DTO>();
client.Indexes.Create(new Index
{
    Name = indexName,
    Fields = fields
});

How can I index my dictionary?

sirdank
  • 3,351
  • 3
  • 25
  • 58

1 Answers1

2

Azure Cognitive Search doesn't support fields that behave like loosely-typed property bags like dictionaries. All fields in the index must have a well-defined EDM type.

If you don't know the set of possible fields at design-time, you have a couple options, but they come with big caveats:

  1. In your application code, add new fields to the index definition as you discover them while indexing documents. Updating the index will add latency to your overall write path, so depending on how frequently new fields are added, this may or may not be practical.
  2. Model your "dynamic" fields as a set of name/value collection fields, one for each desired data type. For example, if a new string field "color" is discovered with value "blue", the document you upload might look like this:
{
    "id": "123",
    "someOtherField": 3.5,
    "dynamicStringFields": [
        {
            "name": "color",
            "value": "blue"
        }
    ]
}

Approach #1 risks bumping into the limit on the maximum number of fields per index.

Approach #2 risks bumping into the limit on the maximum number of elements across all complex collections per document. It also complicates the query model, especially for cases where you might want correlated semantics in queries.

Bruce Johnston
  • 8,344
  • 3
  • 32
  • 42
  • I need to support arbitrary data from end users so what I have now is `new Field("Data", DataType.Collection(DataType.String)) { IsFacetable = true }` and I'm serializing the key value pairs into a single string like this: `(KEY): (VALUE)` which I don't believe will count against the limit on total fields. Although it's not ideal, I don't expect to have even close to 3000 per document so I believe that means I'm safe. Thank you! – sirdank Nov 11 '19 at 13:47
  • The 3000 limit applies only to complex collections (arrays of objects), not collections of strings, so your approach should be fine as long as it supports the query scenarios that you need. – Bruce Johnston Nov 11 '19 at 18:04