0

We have a complex web application that uses SignalR to communicate with clients sending them updates about video encoders. It sends them snapshots of camera images, status of recordings, schedule information of the record, etc every few seconds.

Now from time to time the Application is recycled after the following exception in the routine which fetches the information from the encoder and pushes it to the subscribed clients:

    Core.Assemblers.ScheduleAssembler,Error,GetRecordingInfo failed,"System.Net.Http.HttpRequestException: Response status code does not indicate success: 503 (Service Unavailable).
   at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode()
   at Core.Clients.EncoderClient.<Request>d__25.MoveNext() in e:\jenkins\workspace\sources\Core\Clients\EncoderClient.cs:line 241
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Core.Clients.EncoderClient.<GetRecordingInfo>d__20.MoveNext() in e:\jenkins\workspace\sources\Core\Clients\EncoderClient.cs:line 162
--- End of stack trace from previous location where exception was thrown ---
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at Core.Services.EncoderInfoService.<TryGetClientInfo>d__22.MoveNext() in e:\jenkins\workspace\sources\Services\EncoderInfoService.cs:line 272"~

I'm not certain if threads from SignalR are associated with a Request or not. If not than this exception would tear down the application and explain why the pool is recycled at that time.

In this article it is confirmed that the application pool is recycled after an exception that is not tied to a Reqeust.

My question is. Is this the cause of the pool recycling?

Paul Sinnema
  • 2,534
  • 2
  • 20
  • 33

2 Answers2

0

No, Exceptions in this code do not tear the Application Pool down. I cannot reproduce the problem locally and it still break on the production server. I've deliberately added a throw Exception to the suspect code but the Application is not torn down at any time. I'm giving it a rest for now.

Paul Sinnema
  • 2,534
  • 2
  • 20
  • 33
0

Signalr communication failures do not affect the application process. They definitely don't tear down the app pool. They are handled and logged. You could enable SignalR tracing on your Production server and see if you get clues.

Are you also sure you are not hitting the 5000 concurrent web requests limit or the 1000 request queue limit? Beyond that you might be thrown 503s with the app pool refusing any further connections.

In my opinion, if you are sure that the recycles happen only after the mentioned call, it is a lot more likely that your Encoder code has launched a Task which is throwing an unhandled exception, thereby unloading the App pool.

KRoy
  • 345
  • 1
  • 10