2

I'm new to Azure Stream Analytics. I have an Event hub as input source and now I'm trying to execute a simple query on this stream. An example query is like this:

SELECT
count(*)
INTO [output1]
FROM
[input1] TIMESTAMP BY Time
GROUP BY TumblingWindow(second, 10)

So I want to count the events which arrived within a certain time frame.

When executing this query, I always get the following error:

Request exceeded maximum allowed size limit

As I already narrowed down the checked time window and I'm certain that the amount of events within this time frame is not very big (at most several 100) I'm not sure how to avoid this error.

Do you have a hint?

Thanks!

GANdalf85
  • 129
  • 1
  • 7

1 Answers1

2

Request exceeded maximum allowed size limit

This error(i believe it should be more explicit) indicates that you violated the azure stream analytic resource and object limits.

enter image description here

It's not just about quantity, it's also about size.Please check your source inputs' size or try to reduce the windowsize and test again.

1.Does the record size of the source query mean that one event can only have 64 KB or does this parameter mean 64 K events?

It means the size of one event should below 64KB.

  1. Is there a possibility to use Stream Analytics to select only certain subfields of the event or is the only way to reduce the event size before it is sent to the event hub?

As i know,ASA only collects data for processing it,so the size is all depends on the source side and your query sql. Since you need to use COUNT, i'm afraid that you have to do something on the eventhub side.Please refer to my thoughts:

Use Event Hub Azure Function Trigger, when an event streams into event hub,trigger the function and pick only partial key-values and save it into another event hub namespace.(Just in order to reduce the size of source event) Anyway you only need to COUNT records, i think it works for you.

Jay Gong
  • 23,163
  • 2
  • 27
  • 32
  • Thanks for your reply. It is true, that the events are rather big on which I try to execute the query. An event has about 400 KB. Is there a possibility to use Stream Analytics to select only certain subfields of the event or is the only way to reduce the event size before it is sent to the event hub? Does the record size of the source query mean that one event can only have 64 KB or does this parameter mean 64 K events? – GANdalf85 Oct 31 '19 at 08:48
  • The linked limits refer to Analysis Service which is not the same as Stream Analytics. Stream Analytics operate perfectly well on records that exceed 64KiB in size (we do that all the time). A 64KiB limitation would be weird since ASA is designed around Event Hub inputs and Event Hubs support a message size of up to 1MiB which would make ASA pretty much useless if it imposed a much lower limit. – ChrisWue May 11 '23 at 01:06