How to read a large JSON file as input for Azure Stream Analytics

Question

I am providing a JSON file of 5 MB size as input for Azure Stream Analytics and i am getting the below error.

"selected file 'filename' must be between 1 and 2097152 bytes"

Number of streaming units selected is 3. How to make Stream Analytics to read a large Json file as input?

score 1 · Accepted Answer · answered May 08 '19 at 09:14

1

I think it's limitation for the test of ASA job,you can upload the json file in the Query Editor below 2M only.

Based on the statements in the official document.

For scenarios with large quantities of unstructured data to store in the cloud, Azure Blob storage offers a cost-effective and scalable solution. Data in Blob storage is usually considered data at rest; however, blob data can be processed as a data stream by Stream Analytics.

I suggest you storing large size json file into blob storage and it will be processed as data stream.

answered May 08 '19 at 09:14

Jay Gong

23,163
2
27
32

Thanks @Jay Gong. will try adding the json file into blob storage. – Antony May 08 '19 at 09:30
@Antony Good,if it works,you could mark the answer for others' reference,thanks. – Jay Gong May 08 '19 at 09:51
I confirm that it's a limitation of the testing experience. I'll ask the team to update the error message to make this clearer. Thanks for the feedback. – Jean-Sébastien May 17 '19 at 16:48

score 0 · Answer 2 · answered May 14 '19 at 10:57

0

Storing the large size json file into Blob storage and feeding as input to Stream analytics works. Thanks @Jay Gong

answered May 14 '19 at 10:57

Antony

970
3
20
46

How to read a large JSON file as input for Azure Stream Analytics

2 Answers2