Elasticsearch document ttl not working

Question

I am following the examples given in the documentation to add ttl for documents in elasticsearch: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/docs-index_.html#index-ttl

Using the Sense tool on Chrome, I tried the following and expecting the documents to dissappear in 5 seconds:

PUT /twitter/tweets/2
{
    "_ttl" : "5000",
    "user" : "Romonov",
    "TestField" : "TestData2"
}

PUT /twitters/tweetsy/1?ttl=5000
{
    "user" : "Romonov",
    "TestField" : "TestData1"
}

None of the above are working and the documents are still visible after 5 seconds. I have also tried to set enable _ttl before creating any documents on that index:

PUT /twig/twigsy/_mapping?pretty
{
    "user" : {"_ttl": {"enabled": true}}
}

where, I havent yet PUT any documents on the index twig. But this comes back with an error:

{
   "error": "IndexMissingException[[twig] missing]",
   "status": 404
}

I tried the same with curl(installed it on my windows machine) but getting the same error:

C:\WINDOWS\system32>curl -XPUT "http://localhost:9200/facebook/fb/_mapping?pretty" -d "{ "user" : {"_ttl": {"enabled": true}}"
{
  "error" : "IndexMissingException[[facebook] missing]",
  "status" : 404
}

Wondering what I am missing.

You are using different indexes, and don't you understand error? It say index `twig` and `facebook` is not created yet. — progrrammer, Aug 22 '14 at 05:02
To answer your question, the reason I tried twig and facebook is because I already tried with existing indexes(twitter and twitters) and it did not work. So I was trying the same with new indexes. I even mentioned that in my question. — Romonov, Aug 22 '14 at 14:22

score 2 · Answer 1 · answered Aug 22 '14 at 02:39

The TTL documentation explains this:

Expired documents will be automatically deleted regularly. You can dynamically set the indices.ttl.interval to fit your needs. The default value is 60s.

So docs aren't deleted immediately when their TTL is reached, only when the next expiration task runs. You can lower the rate at which that job runs, but there is a performance hit for doing it.

score 2 · Accepted Answer · answered Aug 25 '14 at 22:23

I could get it to work by doing two things:
1. Add this line in elasticsearch.yml file:

indices.ttl.interval: 7d

2. Create a default-mapping.json document in the same location as elasticsearch.yml with the following lines:

{
    _default_ : { 
        "_ttl" : { 
            "enabled" : true, "default" : "7d" 
            }
        }
}

All new clusters created after doing these two things have ttl enabled to be 7days. In my observation till now it applies to all indices created on these new clusters.

Elasticsearch document ttl not working

2 Answers2