Elasticsearch.Net query using NEST and nested objects to retrieve documents

Question

I'm using Elasticsearch.Net and NEST in an applicationand having trouble accessing documents in an Elasticsearh index when searching based on nested object Ids. The data structure is invoice -> lineItems -> rowItems. I want to search based on these rowItems Id. The (simplified) mapping of the index is:

"invoice": {
    "properties": {
      "lineItems": {
        "properties": {
          "accountId": {
            "index": "not_analyzed",
            "type": "string"
          },
          "listItems": {
            "properties": {
              "itemName": {
                "analyzer": "str_index_analyzer",
                "term_vector": "with_positions_offsets",
                "type": "string",
                "fields": {
                  "raw": {
                    "analyzer": "str_search_analyzer",
                    "type": "string"
                  }
                }
              },
              "listItemID": {
                "index": "not_analyzed",
                "type": "string"
              }
            }
          }
        }
      }
    }
}

And when I do a sense search in chrome of one of the nested objects I can successfully retrieve it:

POST /_search
{
  "query": {
    "bool": {
      "should": [
        {"match": {
            "lineItems.rowItems.rowItemID" : "23f2157f-eb21-400d-b3a1-a61cf1451262"            
        }} 
      ]
    }
  }
}

Which returns the document type Invoice with all its details.

I've been playing around with the code to do this using NEST but have failed so far. I have a list of rowItemIds and I want to get all invoice documents that have an exact match on those Ids. This is what I currently have:

            var result = Execute(client => client.Search<Invoice>(s => s
    .Aggregations(a => a
        .Nested("my_nested_agg", n => n
            .Path("lineItems")
            .Aggregations(aa => aa
                .Filter("my_avg_agg", avg => avg
                    .Field(p => searchIds.Contains(p.LineItems.First().RowItems.First().TrackingItemID))
                )
            )
        )
    )
));

Where searchIds is the list of rowItemIds I'm searching for. The above code is totally wrong and I'm not familiar with the syntax on how to do this. Any help would be greatly appreciated.

Answer 1

Nested types are needed in scenarios where you wish to query across properties of a object property. Given your example,

1. If you want to only query the listItemID of listItems on lineItems then having an object type for this will work fine.

2. If you want to query the listItemID and itemName of listItems on lineItems, you would need to map listItems as a nested type.

The reason for this is that without using nested type, the association between the properties of a particular listItem are not stored when indexed. With a nested type, the association is stored (nested types are internally stored as documents).

The search query that you have is pretty similar in NEST; the match query doesn't need to be contained in a bool query should clause in this case

var client = new ElasticClient();

var searchResponse = client.Search<Invoice>(s => s
    .AllIndices()
    .AllTypes()
    .Query(q => q
        .Match(m => m
            .Field(f => f.LineItems.First().ListItems.First().ListItemID)
            .Query("23f2157f-eb21-400d-b3a1-a61cf1451262")
        )
    )
);

The lambda expression to get the field name is just that; an expression to get the field name.

This generates the query

POST http://localhost:9200/_search
{
  "query": {
    "match": {
      "lineItems.listItems.listItemID": {
        "query": "23f2157f-eb21-400d-b3a1-a61cf1451262"
      }
    }
  }
}

since listItemID is a not_analyzed string field, you can use term query here instead and, since you probably don't need a score calculated (a match in this case is either true or false), you can wrap this in a bool query filter clause which can take advantage of filter caching and should perform slightly better.

To get those documents that match a collection of ids, we can use the terms query

var ids = new[] {
    "23f2157f-eb21-400d-b3a1-a61cf1451262",
    "23f2157f-eb21-400d-b3a1-a61cf1451263",
    "23f2157f-eb21-400d-b3a1-a61cf1451264"
};

var searchResponse = client.Search<Invoice>(s => s
    .AllIndices()
    .AllTypes()
    .Query(q => q
        .Terms(m => m
            .Field(f => f.LineItems.First().ListItems.First().ListItemID)
            .Terms(ids)
        )
    )
);

And finally, a shorthand for wrapping this in a bool query filter clause, using the unary + operator

var searchResponse = client.Search<Invoice>(s => s
    .AllIndices()
    .AllTypes()
    .Query(q => +q
        .Terms(m => m
            .Field(f => f.LineItems.First().ListItems.First().ListItemID)
            .Terms(ids)
        )
    )
);