Strange error in SQL Server from ASP.NET app

Question

I posted this question on stack overflow, but to no avail. Getting desperate here.

So we have an extremely random error (by which I mean, there appears to be no usage pattern that precedes the error) in our C# ASP.NET MVC5 application, that occurs in our production environment.

The error appears to occur only sometimes after a user logs in, and the entire system crashes and becomes inaccessible to all users until we manually go into IIS8 and restart the web server. We are running SQL Server Express 2012 SP2 on a windows server 2012 environment with IIS.

The error message as follows:

IndexOutOfRangeException/Account/Login
Index was outside the bounds of the array.

The stack trace is as follows:

IndexOutOfRangeException·Index was outside the bounds of the array.
Raw
:0System.Data.SqlClient.SqlDataReader.CheckHeaderIsReady(Int32 columnIndex, Boolean permitAsync, String methodName) 
 :0System.Data.SqlClient.SqlDataReader.IsDBNull(Int32 i)    
 :0System.Data.Entity.Core.Common.Internal.Materialization.Shaper+ErrorHandlingValueReader`1.GetValue(DbDataReader reader, Int32 ordinal)   
 :0lambda_method(Closure , Shaper ) 
 :0System.Data.Entity.Core.Common.Internal.Materialization.Shaper.HandleEntityAppendOnly(Func`2 constructEntityDelegate, EntityKey entityKey, EntitySet entitySet)  
 :0lambda_method(Closure , Shaper ) 
 :0System.Data.Entity.Core.Common.Internal.Materialization.Coordinator`1.ReadNextElement(Shaper shaper) 
 :0System.Data.Entity.Core.Common.Internal.Materialization.Shaper`1+SimpleEnumerator+<MoveNextAsync>d__4.MoveNext() 
 :0System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
 :0System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   
 :0System.Data.Entity.Internal.LazyAsyncEnumerator`1+<FirstMoveNextAsync>d__0.MoveNext()    
 :0System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
 :0System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   
 :0System.Data.Entity.Infrastructure.IDbAsyncEnumerableExtensions+<FirstOrDefaultAsync>d__25`1.MoveNext()   
 :0System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
 :0System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   
 :0Microsoft.AspNet.Identity.TaskExtensions+CultureAwaiter`1.GetResult()    
 :0Microsoft.AspNet.Identity.EntityFramework.UserStore`6+<GetUserAggregateAsync>d__6c.MoveNext()    
 :0System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
 :0System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   
 :0Microsoft.AspNet.Identity.TaskExtensions+CultureAwaiter`1.GetResult()    
 :0Microsoft.AspNet.Identity.UserManager`2+<FindAsync>d__12.MoveNext()  
 :0System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
 :0System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   
 :0Saleboat.Controllers.AccountController+<Login>d__9.MoveNext()    
 :0System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)    
 :0System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)   
 :0System.Web.Mvc.Async.TaskAsyncActionDescriptor.EndExecute(IAsyncResult asyncResult)  
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass37.<BeginInvokeAsynchronousActionMethod>b__36(IAsyncResult asyncResult)   
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethod(IAsyncResult asyncResult)    
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3d()    
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f()    
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker+AsyncInvocationWithFilters+<>c__DisplayClass46.<InvokeActionMethodFilterAsynchronouslyRecursive>b__3f()    
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker.EndInvokeActionMethodWithFilters(IAsyncResult asyncResult) 
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass21+<>c__DisplayClass2b.<BeginInvokeAction>b__1c() 
 :0System.Web.Mvc.Async.AsyncControllerActionInvoker+<>c__DisplayClass21.<BeginInvokeAction>b__1e(IAsyncResult asyncResult)

This error only occurs on our live server/database and cannot be intentionally reproduced, we pretty much have to wait for it to happen and review the exception caught by Bugsnag.

EDIT

Here is the code executed upon sign in:

    [HttpPost]
    [AllowAnonymous]
    [ValidateAntiForgeryToken]
    public async Task<ActionResult> Login(LoginViewModel model, string returnUrl)
    {
        if (ModelState.IsValid)
        {
            var user = await UserManager.FindAsync(model.UserName, model.Password);
            if (user != null)
            {
                var userId = UserManager.FindByName(model.UserName).Id;
                int companyId = db.Users.Find(userId).CompanyId;
                await SignInAsync(user, model.RememberMe);
                return RedirectToLocal(returnUrl);
            }
            else
            {
                ModelState.AddModelError("", "Invalid username or password.");
            }
        }
        return View(model);
    }

Have you tried putting a try/catch around that section? If it comes to it, you could try/catch each line individually to create a log for _exactly_ where the exception is thrown. — Joel Coel, Mar 15 '16 at 17:38
I don't know if that will help, i mean we do use bugsnag which catches everything, that's the only reason i even know what happens — barnacle.m, Mar 15 '16 at 17:42
Well... right now you only know you have a problem in this method. Adding try/catch blocks can tell you which line blows up inside the method. Admittedly the real problem is often a line or three before the exception, but this could still help you narrow your focus. — Joel Coel, Mar 15 '16 at 17:46
Best guess, though, is the problem is here: `db.Users.Find(userId).CompanyId`, given the stack trace shows `SqlClient` a lot. That we also see an `IsDbNull()` call in the trace makes me wonder if someone has a disconnect between their ASP.Net Identity username from `UserManager` and the username stored in the database. — Joel Coel, Mar 15 '16 at 17:48
I'm actually going to remove that line of code anyway, it isn't needed anymore, as well as add a try/catch. — barnacle.m, Mar 15 '16 at 17:58
Yes. Upgrade to the latest .net core LTS (3.1 at the moment), upgrade to entity framework core and get the hell away from IIS. — barnacle.m, Jul 07 '20 at 10:21

Strange error in SQL Server from ASP.NET app

0 Answers0