Recently we faced a situation where our app service plan CPU was at 100% for considerable amount of time (8-10 hours). Our app service was using B2 plan (development), scale-out 2, SQL server (250GB) 20 DTU.
Our application hosted in the app service plan crashed with error "The request limit for the database is 90 and has been reached" as it receives lots of requests during that time.
In order to resolve the issue quickly, we upscaled our app-service plan to P3 (production) increased the scale out to 6, DTU to 1600, after this our application started working again but still some functionalities were not working as one of the app service was constantly at 100% CPU usage we tried increasing scale out to 16 still same behavior, after some hours when request count drops significantly the CPU count was still at 100 %.
We ran the kudu profiler for 3-4 minutes and checked the diagnosis, there was an entry for entity framework which was consuming the most CPU around 90%, but we were not using entity framework anywhere in our application code.
Further debugging leads to session state used in the application (which uses Entity framework in behind the scene).
Below is the web.config for setting for session state
<sessionState mode="Custom" customProvider="DefaultSessionProvider" timeout="30">
<providers>
<clear />
<add name="DefaultSessionProvider" type="System.Web.Providers.DefaultSessionStateProvider, System.Web.Providers, Version=2.0.0.0, Culture=neutral, PublicKeyToken=35bg3856ae324e35" connectionStringName="azuredbconstring" />
</providers>
</sessionState>
as a part of quick resolution we changed session state mode to inproc, and suddenly everything goes normal, cpu was at 0-1% now.
We changed app service plan to B2 again, scale out to 2, database DTU to 20, still all is good, CPU is at 0-1% only.
Then we observed that in the sessions table there were 30000 records (individual record size almost 14K) all created that same day, none of the entries were deleted post expiration.
Can somebody please explain why auto clean-up didn't worked?
We checked performance insight (azure =>database=> performance overview => query performance insight) and observed below Query was consuming the most CPU
SELECT
[Extent1].[SessionId] AS [SessionId],
[Extent1].[Created] AS [Created],
[Extent1].[Expires] AS [Expires],
[Extent1].[LockDate] AS [LockDate],
[Extent1].[LockCookie] AS [LockCookie],
[Extent1].[Locked] AS [Locked],
[Extent1].[SessionItem] AS [SessionItem],
[Extent1].[Flags] AS [Flags],
[Extent1].[Timeout] AS [Timeout]
FROM [dbo].[Sessions] AS [Extent1]
WHERE [Extent1].[Expires] < (SysUtcDateTime())
We are assuming this is the query used by auto clean up to elect and delete.
We reproduced the scenario by copying sessions table data in test environment, CPU went to 100% within minutes, data not deleted from test environment as well.
How auto clean up works in azure environment?
