2

I am streaming data from devices and I want to use the LAG function to identify the last value received from a particular device. The data is not streamed at a regular period and in rare cases it could be days between receiving data from a device.

Is there a maximum period for the LIMIT DURATION clause?

Is there any down-side to having long LIMIT DURATION periods?

Brad
  • 21
  • 2

1 Answers1

3

There is no maximum period for LIMIT DURATION in the language. However it is limited by amount of data the input source can hold - e.g. 1 day is default retention policy for Event Hub (can be increased in configuration).

When job is being started, Azure Stream Analytics reads up to LIMIT DURATION amount of data from the source to make sure it has correct value for the LAG at job start time. If data volume is high, this can increase job start time.

If you need to use data that is more than several days old, it may make more sense to use it as a reference data (which can be updated at daily intervals for example)

Konstantin Zoryn
  • 601
  • 4
  • 3