How to reproduce the ElasticSearch query?

Question

I have ElasticSearch query with which I fetch data through Kibana. I also require the same request in my Java application. The query generated by Kibana is the following one:

{
  "version": true,
  "size": 500,
  "sort": [
    {
      "@timestamp": {
        "order": "desc",
        "unmapped_type": "boolean"
      }
    }
  ],
  "_source": {
    "excludes": []
  },
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "@timestamp",
        "fixed_interval": "30s",
        "time_zone": "Europe/Berlin",
        "min_doc_count": 1
      }
    }
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "@timestamp",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "query_string": {
            "query": "ORDERID=120019",
            "analyze_wildcard": true,
            "time_zone": "Europe/Berlin"
          }
        }
      ],
      "filter": [
        {
          "match_phrase": {
            "service": {
              "query": "some-service-app"
            }
          }
        },
        {
          "range": {
            "@timestamp": {
              "format": "strict_date_optional_time",
              "gte": "2020-02-24T09:12:41.685Z",
              "lte": "2020-02-24T09:27:41.685Z"
            }
          }
        }
      ],
      "should": [],
      "must_not": []
    }
  },
  "highlight": {
    "pre_tags": [
      "@kibana-highlighted-field@"
    ],
    "post_tags": [
      "@/kibana-highlighted-field@"
    ],
    "fields": {
      "*": {}
    },
    "fragment_size": 2147483647
  }
}

I am trying to compose the same query using ElasticSearch Java API but getting completely different results that don't match the expected output at all.

Could you please help me to come up with a correct one? The way I'm doing it now is below.

    final QueryBuilder query = QueryBuilders.boolQuery()
            .must(QueryBuilders.simpleQueryStringQuery("some-service-app").field("service"))
            .must(QueryBuilders.simpleQueryStringQuery("INFO").field("severity"))
            .must(QueryBuilders.rangeQuery("@timestamp").from(now.minusDays(15)))
            .must(QueryBuilders.simpleQueryStringQuery("ORDERID=120019"));
    final SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();
    sourceBuilder.query(query);
    final SearchRequest searchRequest = new SearchRequest(targetIndexName);
    searchRequest.source(sourceBuilder);
    final SearchResponse search = client.search(searchRequest, RequestOptions.DEFAULT);

Unfortunately, my code returns wrong results even with wrong "service" values.

Could you please help me to align my Java request with the JSON one?

You can check the search-json generated by your java code and compare it , where it differs — Amit, Feb 24 '20 at 10:05
@OpsterESNinja but I'm not an expert to spot the deviation quickly. I just need to query based on the value of one field "service" and "ORDERID=" string across the entire document. These are 2 filters that I expect easy to apply but I do smth wrongly. — Pasha, Feb 24 '20 at 10:39

LeBigCat · Answer 1 · 2020-02-24T13:31:03.907

1) The score wont be the same. Filter doesn't count in a score, but in your java code, you use query instead, so the sort would be different.

read: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-filter-context.html

2) I think you doesn't want to reproduce exactly the kibana's query. For example I didn't found "severity" field in the json, and the aggregation part is missing from your java, what s are you looking for exactly?.... Please share your mappings

3) Please compare the datetime between kibana and your documents timestamp. Sometimes with timezone, kibana shows different hours.

4) as said in comments, you must have the generated JSON when you debug, this could help you.

How to reproduce the ElasticSearch query?

1 Answers1