I'm using Microsoft.AnalysisServices.Tabular to test refresh / loading data into azure analysis services. Code is quite straightforward
while (true)
{
var server = new Microsoft.AnalysisServices.Tabular.Server();
server.Connect(
"Provider=MSOLAP;Data Source=asazure://[...];Persist Security Info=True; Impersonation Level=Impersonate;");
var m = server.Databases[0].Model;
var tabs = m.Tables;
tabs["Viewer"].RequestRefresh(RefreshType.DataOnly);
tabs["EventClick"].RequestRefresh(RefreshType.DataOnly);
[...]
m.SaveChanges();
server.Disconnect();
Thread.Sleep(new TimeSpan(0,0,0,1));
}
Model initially is around 300MB, doesn't have active relationships or complicated measures. Basically it's just data that grows very slowly (less than 10 records per second in all tables combined).
I expected memory utilization to grow a bit and stabilize around some number but for some reason memory required by database just grows and grows (at constant rate) with each refresh till it reaches azure memory limits for azure AS dev plan (around 3GB). At that point apllication receives exception
Unhandled Exception: Microsoft.AnalysisServices.OperationException: Failed to save modifications to the server. Error returned: 'Memory error: Allocation failure : The paging file is too small for this operation to complete.
Memory utilization remains constant even when i stop refreshing tables / partitions. I've noticed that it goes down to original value of ~300MB when i pause and restart azure AS in azure portal
DMV results doesn't tell me anything but i'll post it anyway (snapshot, where db is around 1.5GB). Maybe someone will notice something odd
DMV: SELECT top 10 * FROM $SYSTEM.DISCOVER_OBJECT_MEMORY_USAGE ORDER BY OBJECT_MEMORY_NONSHRINKABLE DESC
OBJECT_PARENT_PATH OBJECT_ID OBJECT_MEMORY_SHRINKABLE OBJECT_MEMORY_NONSHRINKABLE OBJECT_VERSION OBJECT_DATA_VERSION OBJECT_TYPE_ID OBJECT_TIME_CREATED OBJECT_MEMORY_CHILD_SHRINKABLE OBJECT_MEMORY_CHILD_NONSHRINKABLE OBJECT_GROUP
"Global" "Big" 0 11026432 "-1" "-1" "2" 2016-12-13 12:01:26.000 0 0 "$System"
"" "Global" 0 5055222 "-1" "-1" "100011" 2016-12-13 12:01:26.000 0 15903456 "$System"
"Global" "SystemHeap" 0 1396488 "-1" "-1" "2" 2016-12-13 12:01:26.000 0 0 "$System"
"ardev.Databases.mytestdb_90db7154-080a-4094-bd4b-cc6b35422d42.Dimensions.ProductImpression (28).In-Memory Table.Columns" "ProductId (73)" 0 1068088 "0" "-1" "703003" 2016-12-13 04:00:52.000 0 528 ""
"ardev.Databases.mytestdb_90db7154-080a-4094-bd4b-cc6b35422d42.Dimensions.Website Product Sale (31).In-Memory Table.Columns" "ProductId (78)" 0 1066288 "-1" "-1" "703003" 2016-12-13 04:00:52.000 0 136 ""
"Global" "IMBIData" 0 880640 "-1" "-1" "2001" 2016-12-13 12:01:26.000 0 0 "$System"
"Global" "Aligned Global Heap" 0 619488 "0" "-1" "2" 2016-12-13 12:01:26.000 0 0 "$System"
"ardev.Databases.mytestdb_90db7154-080a-4094-bd4b-cc6b35422d42.Dimensions.Visit (16).In-Memory Table.Columns" "Url (56)" 0 508404 "-1" "-1" "703003" 2016-12-13 04:00:52.000 0 22040 ""
"Global" "GeneralPurpose" 0 413480 "-1" "-1" "2" 2016-12-13 12:01:26.000 0 0 "$System"
"Global" "Allocators" 0 404176 "-1" "-1" "2" 2016-12-13 12:01:26.000 0 0 "$System"
What is causing this "memory leaks" and how to avoid them ? Are the any gotchas i should know about ?
// Edit
What happens when i refresh less often ? Currently we refresh data each 5 minutes and restart instance every 3 hours.
Memory metric look like this:
Basically problem is still visible when you refresh every x minutes. I assume most people that refresh data every x days would not be able to tell that something is wrong.
