Memory limit in Azure Data Lake Analytics

Question

I have implemented a custom extractor for NetCDF files and load the variables into arrays in memory before outputting them. Some arrays can be quite big, so I wonder what the memory limit is in ADLA. Is there some max amount of memory you can allocate?

saveenr · Accepted Answer · 2018-04-08T22:08:09.557

5

Each vertex has 6GB available. Keep in mind that this memory is shared between the OS, the U-SQL runtime, and the user code running on the vertex.

edited Apr 08 '18 at 22:08

answered Dec 15 '16 at 13:40

saveenr

8,439
3
19
20

score 2 · Answer 2 · answered Dec 16 '16 at 17:16

2

In addition to Saveen's reply: Please note that a row can at most contain 4MB of data, thus your SqlArray will be limited by the maximal row size as well once you return it from your extractor.

answered Dec 16 '16 at 17:16

Michael Rys

6,684
15
23

Memory limit in Azure Data Lake Analytics

2 Answers2