0

We want to run logstash in an ECS environment where the instances have transient lives. They might be long lived, but they aren't permanent. Thus, having logstash store the sql_last_value in the last_run_metadata_path doesn't work if the instance is removed and then brought up again. That state gets lost.

Is there a way to have that value pulled from a S3 path? Or via a DB query? I'm just looking somewhere persistent that won't vanish when the instance gets deleted and brought up again.

last_run_metadata_path => "/home/ckessel/.logstash_source_a"  <-- need that to be S3 or some such
Chris Kessel
  • 5,583
  • 4
  • 36
  • 55

1 Answers1

1

The file input just reads the saved value from a file. So the location can be anything that the Ruby IO::read function can read. That does not include s3 or a database. It needs to be a filesystem, but it can be network mounted.

Badger
  • 3,943
  • 2
  • 6
  • 17