We have an airflow job that captures data from an API every 15 mins. Can I store this time series data in Data Warehouse? It's not fit the grain of any of our star schemas but will be used with the aggregated results from star schema when used with the BI tools. Is this the right place to store it?
Asked
Active
Viewed 168 times
0
-
This is a use case for a "Data Lake", which used to be called an "ODS". It's data storage that reflects the source system with minimal transformation and modelling. It doesn't join very nicely to the star schema but you should be able to drill through to it from data in the star schema – Nick.Mc Dec 08 '21 at 23:43
1 Answers
0
Many Data Warehouse Solutions like Cloudera or Snowflake can support the most demanding time series analytics needs. They have the capability to handle everything from real-time data ingestion at scale to enriching the data from the data warehouse to making easily consumable visualizations.
Amira Bedhiafi
- 8,088
- 6
- 24
- 60