1

I have a Kusto table, which has incoming streaming data. However, I need to store the aggregates per day in another table. Is there a way to run scheduled functions in Kusto which for example runs at midnight and ingests the result into an aggregate table? If no, how can I achieve my goal? Are there some connectors available with which I can achieve this?

anna
  • 405
  • 5
  • 13

1 Answers1

5

Materialized views is the ideal solution for it. Please note that materialized views over table with streaming data is now in private preview so you will need to open a support ticket for it. See the applicable comment in "create materialized views" document:

Materialized views over streaming ingestion tables are supported in preview mode. Enabling this feature on your cluster requires creating a support ticket.

Avnera
  • 7,088
  • 9
  • 14
  • I read we can use power-automate connector to run scheduled queries in Kusto and visualize results. Can I use this to run scheduled query and ingest it to the aggregate table? – anna Jun 21 '21 at 03:57
  • Yes, you can use it as well, see the walkthrough here: https://learn.microsoft.com/en-us/azure/data-explorer/flow. This approach however has significant drawbacks compared with materialized views, mostly around the need to handle edge cases such as late-arriving data and failures. Thus it is highly recommended to use the materialized views approach. – Avnera Jun 21 '21 at 08:37
  • Since the materialized view is in preview mode, is it good enough for the production environment? And also is using an azure function a workaround? Like by using .net SDK for Kusto, query Kusto first, get the results, and ingest into another table in Kusto in the same function? – anna Jun 21 '21 at 12:30
  • Materialized views are already GA feature, the preview part is the materialized views over streaming data. My personal opinion is that the challenges of implementing the function approach in production are far greater than monitoring the materialized view approach and opening a support ticket in case you run into a bug. – Avnera Jun 21 '21 at 13:31
  • Got it. Thank you very much. – anna Jun 21 '21 at 18:29