1

I am building out a simple sensor which sends out 5 telemetry data to AWS IoT Core. I am confused between AWS Timestream DB and Elastic Search to store this telemetries.

  1. For now I am experimenting with Timestream and wanted to know is this the right choice ? Any expert suggestions.
  2. Secondly I want to store the db records for ever as this will feed into my machine learning predictions in the future. Timestream deletes records after a while or is it possible to never delete it
  3. I will be creating a custom web page to show this telemetries per tenant - any help with how I can do this. Should I directly query the timestream db over api or should i back it up in another db like dynamic etc ?

Your help will be greatly appreciated. Thank you.

KPCoders
  • 13
  • 4

2 Answers2

1

For now I am experimenting with Timestream and wanted to know is this the right choice? Any expert suggestions.

I would not call myself an expert but Timestream DB looks like a sound solution for telemetry data. I think ElasticSearch would be overkill if each of your telemetry data is some numeric value. If your telemetry data is more complex (e.g. JSON objects with many keys) or you would benefit from full-text search, ElasticSearch would be the better choice. Timestream DB is probably also easier and cheaper to manage.

Secondly I want to store the db records for ever as this will feed into my machine learning predictions in the future. Timestream deletes records after a while or is it possible to never delete it

It looks like the retention is limited to 4 weeks 200 Years per default. You probably can increase that by contacting AWS support. But I doubt that they will allow infinite retention.

We use Amazon Kinesis Data Firehose with AWS Glue to store our sensor data on AWS S3. When we need to access the data for analysis, we use AWS Athena to query the data on S3.

I will be creating a custom web page to show this telemetries per tenant - any help with how I can do this. Should I directly query the timestream db over api or should i back it up in another db like dynamic etc ?

It depends on how dynamic and complex the queries are you want to display. I would start with querying Timestream directly and introduce DynamoDB where it makes sense to optimize cost.

linuxbasic
  • 36
  • 4
  • 1
    Your link appears to have updated. Retention in the magnetic store appears to be 73000 hours both by default and at maximum, equivalent to 200 years. The in-memory store is by default 6 hours, with a maximum of 8766 hours or 365.25 days. – ESilk Oct 12 '21 at 14:43
0

Based on your approach " simple sensor which sends out 5 telemetry data to AWS IoT Core" Timestream is the way to go, fairly simple and cheaper solution for simple telemetry data.

The Magnetic storage is above what you will ever need (200years)

Panagiotis Giannelos
  • 86
  • 7