The end goal is to have logs in Loki for 1 year.
The configuration of the instance is as follows:
schema_config:
configs:
- from: 2020-10-24
store: boltdb-shipper
object_store: filesystem
schema: v11
index:
prefix: index_
period: 24h
compactor:
working_directory: /data/loki/boltdb-shipper-compactor
shared_store: filesystem
retention_enabled: True
retention_delete_delay: 2h
retention_delete_worker_count: 150
limits_config:
enforce_metric_name: false
reject_old_samples: false
reject_old_samples_max_age: 8760h
retention_period: 8760h
max_query_length: 1000h
split_queries_by_interval: 15m
chunk_store_config:
max_look_back_period: 8760h
table_manager:
retention_deletes_enabled: True
retention_period: 8760h
After the change Loki was restarted, the service is running. No errors are observed and logs are ingested and can be queried. However the logs retained for 31 days instant of 365(or 8760 hours).
In journalctl, Loki outputs the following logs related to retention:
Jul 07 07:26:31 level=info ts=2023-07-07T05:26:31.569976084Z caller=compactor.go:454 msg="applying retention with compaction"
Jul 07 07:26:31 level=info ts=2023-07-07T05:26:31.570032777Z caller=expiration.go:74 msg="overall smallest retention period 1657171591.57, default smallest retention period 1657171591.57"
Any ideas why the retention may not be taking place as desired?
I tried the following:
Changing hours to days and days to hours in the config
Doublecheck for cron jobs which may interfere with Loki mechanisms - none were found.
Checked for errors in the Loki service via journalctl - none were found
Reviewed Loki github issue, found a few similar reports none of which had answers.
