C# NEST's serializing FieldValue & AggegateDictionary all nulls

Question

Good day:

I'm querying my Elastic instance and getting back results for both my scriptField and Aggregation:

ScriptField

"fields": {
          "distance": [
            6569.304614953225
          ]
        }

Aggregations

"aggregations": {
    "children#Capacity": {
      "doc_count": 15,
      "histogram#Capacity": {
        "buckets": [
          {
            "key": 0,
            "doc_count": 15
          }
        ]
      }
    },
    "geo_distance#Distance": {
      "buckets": [
        {
          "key": "*-5.0",
          "from": 0,
          "to": 5,
          "doc_count": 15
        },
        {
          "key": "6.0-105.0",
          "from": 6,
          "to": 105,
          "doc_count": 0
        },
        {
          "key": "106.0-505.0",
          "from": 106,
          "to": 505,
          "doc_count": 0
        }
      ]
    }
  }

The Elastic response gets mapped to ISearchResponse however serializing ISearchResponse.Hits.Aggregations and ISearchResponse.Hits.Fields into JSON, the values are all empty:

Fields

{ 
    "fields": {
    "distance": {}
    }
}

Aggregations

 {
    "Aggs": {
    "Capacity": {
    "Capacity": {
    "Items": [
      {}
    ],
    "DocCountErrorUpperBound": null,
    "SumOtherDocCount": null,
    "Meta": null,
    "DocCount": 0,
    "BgCount": 0
    }
    },
    "Distance": {
    "Items": [
      {},
      {},
      {}
    ],
    "DocCountErrorUpperBound": null,
    "SumOtherDocCount": null,
    "Meta": null,
    "DocCount": 0,
    "BgCount": 0
    }
    }
}

As you can see the distance in the scriptField response is empty. Also the items in the Aggregation is also empty.

score 2 · Answer 1 · answered Aug 15 '18 at 00:53

Aggregation response types only have deserialization implementations, not serialization implementations. Check out the discussion in GitHub issue 3093.

If needing to reserialize to a JSON string, you can take one of two approaches

make the request with the low level client, using the high level search request type, and return a string response. Then, if needing a strongly typed response from the JSON string, perform the deserialization to SearchResponse<T> in the application

// build your search request using NEST's request types
var descriptor = new SearchDescriptor<Question>()
    .Query(q => q
        .Match(m => m
            .Field(f => f.Title)
            .Query("Elasticsearch Kibana")
        ) && +q
        .Term("type", "question")
    );

var response = client.LowLevel.Search<StringResponse>("posts", PostData.Serializable(descriptor));

// the JSON string response from Elasticsearch
var jsonString = response.Body;

ISearchResponse<Question> searchResponse = null;
using (var stream = new MemoryStream(Encoding.UTF8.GetBytes(jsonString)))
    searchResponse = client.RequestResponseSerializer.Deserialize<SearchResponse<Question>>(stream);

// do something with the documents
var documents = searchResponse.Documents;

or

set .DisableDirectStreaming() in the request configuration for this request only, so that the response bytes are buffered on the ISearchResponse<T>. Then get the JSON string response from these bytes with Encoding.UTF8.GetString(response.ApiCall.ResponseBodyInBytes);

var searchResponse = client.Search<Question>(s => s
    .Query(q => q
        .Match(m => m
            .Field(f => f.Title)
            .Query("Elasticsearch Kibana")
        ) && +q
        .Term("type", "question")
    )
    .RequestConfiguration(r => r
        // capture the request and response bytes for this call only
        .DisableDirectStreaming()
    )
);

// do something with the documents
var documents = searchResponse.Documents;

// the JSON string response from Elasticsearch
var jsonString = Encoding.UTF8.GetString(searchResponse.ApiCall.ResponseBodyInBytes);

Taking one of these approaches also means that the JSON string represents exactly what has come back from Elasticsearch.

C# NEST's serializing FieldValue & AggegateDictionary all nulls

1 Answers1