0

In continuation to my comments on this SO answer, I have below setup:

Using read heavy ES index which also get index requests but ratio is 25:1 and using refresh_interval of 1 sec, and trying to improve the query performance by explicitly setting this param so that it caches the search query hits as well but not seeing any performance gain.

Also I see hits.total is also changing,As my index is also getting the write request and I feel this could be the cause as shard is getting refreshed and cache is getting invalidated.

Please confirm if my assumption is correct and is there is any way to improve the performance using various cache settings available in ES?

Note I used the monitoring cache section mentioned in https://opensourceconnections.com/blog/2017/07/10/caching_in_elasticsearch/ and below is the O/P I got.

URL:- http://:9200/_cat/nodes?v&h=queryCacheMemory,queryCacheEvictions,requestCacheMemory,requestCacheHitCount,requestCacheMissCount,flushTotal,flushTotalTime

queryCacheMemory queryCacheEvictions flushTotal flushTotalTime 
              0b                   0     353204           1.9h 
              0b                   0          0             0s 
              0b                   0     464814           2.2h 
              0b                   0     292127           1.6h 
              0b                   0     409013           2.1h 
              0b                   0     394303             2h 
              0b                   0     369545           2.1h 
              0b                   0          0             0s 
              0b                   0          0             0s

Please note, even after having requestCacheMemory, hit and miss count of it, its not included in the O/P

2 Answers2

0

The query string parameter request_cache=true is only useful if the request cache is disabled at the index level, but since it's enabled by default, specifying that parameter has no effect unless you've explicitly disabled the cache at the index level.

Furthermore, are you sure that you're always sending the exact same JSON query? The reason I'm asking is because the cache key is created out of the JSON query, so even if a single character changes in the query, the key won't be the same.

Finally, you can check how the request cache is hit or missed by checking out this endpoint

GET your-index/_stats/request_cache?human

What is increasing? the hit_count or the miss_count?

Val
  • 207,596
  • 13
  • 358
  • 360
  • I am using this in old version and it doesn't return any value on request_cache :( –  Sep 14 '20 at 06:34
  • What version are you using? – Val Sep 14 '20 at 06:34
  • 1.4 :( and the document which you mentioned in not available for 1.4, also its not returning any memory and eviction count when checked for several cache param mentioned in https://opensourceconnections.com/blog/2017/07/10/caching_in_elasticsearch/ –  Sep 14 '20 at 06:37
  • I am using the endpoint `_cat/nodes?v&h=id,queryCacheMemory,queryCacheEvictions,requestCacheMemory,requestCacheHitCount,requestCacheMissCount,flushTotal,flushTotalTime` and below is how response looks like –  Sep 14 '20 at 06:38
  • `queryCacheMemory queryCacheEvictions flushTotal flushTotalTime 0b 0 353204 1.9h 0b 0 0 0s 0b 0 464814 2.2h 0b 0 292127 1.6h 0b 0 409013 2.1h 0b 0 394303 2h 0b 0 369545 2.1h` –  Sep 14 '20 at 06:39
  • Not legible in comments... instead, you should update your question with that information – Val Sep 14 '20 at 06:40
  • But if you're really concerned about performance, then the first thing you should do is to upgrade to more recent versions. 1.4 is not supported anymore! – Val Sep 14 '20 at 06:40
  • Let us [continue this discussion in chat](https://chat.stackoverflow.com/rooms/221433/discussion-between-prerna-gupta-and-val). –  Sep 14 '20 at 06:40
  • I know, but this is not in my hand, anything which i can do, I also want to know if there was caching option in this version. –  Sep 14 '20 at 06:40
  • edited the question with required info, please have a look and let me know if I need to provide more info –  Sep 15 '20 at 05:01
  • These are the available columns for the `_cat/nodes` API in 1.4: https://www.elastic.co/guide/en/elasticsearch/reference/1.4/cat-nodes.html#_columns, I don't see `queryCacheMemory` and `queryCacheEvictions` – Val Sep 15 '20 at 05:03
  • not sue, that might be the reason its coming as 0, anyway my point is there is anything which we can tune to cache in this really old version of ES? –  Sep 15 '20 at 05:09
0

Also, why do you keep refresh interval at 1s? Try 30-60s which will lower load and also not create as many segments to be merged - I was looking at merging recently and I think it will invalidate some types of caching as that happens, though maybe not the general query cache.

Steve Mushero
  • 36
  • 3
  • I also need near real time search and index updates, hence can't increase refresh interval value –  Sep 14 '20 at 04:55