Firstly I apologise if this, to you, is obviously a hardware issue, and not a programming issue. If it is, hopefully we can migrate it to the appropriate place.
I have a long-running web-services program (Any CPU), which has a whole bunch of data repositories that are held in-memory. Some are refreshed every hour, others less often. Some of these repositories sort the data (simple POCOs) after retrieving them from 3rd party sources.
Once in a blue moon, I get an AccessViolationException within the method ComputeKeys during a Sort. Some of these repositories use List<T>.Sort after getting the data from the 3rd party sources.
The code that is performing these sorts is trivial and depends on the repository, but here are two examples:
pocos.Sort((a, b) => a.BookingDate.CompareTo(b.BookingDate));
or:
pocos.Sort((a, b) => a.Date.CompareTo(b.Date));
Note that these data-sets are quite big (comparatively). One has over 140,000 elements. I log exceptions to a database, and this is the log I have (The times correlate to another log I have that shows a repository was finishing fetching data at the time):
Time Host Data
05/11/2014 16:50 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:50 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:38 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:29 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:26 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:23 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:21 HOST2 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:19 HOST1 {"Name":"IndexOutOfRangeException","Method":"Dispose","Source":"System.Core","Message":"Index was outside the bounds of the array.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"Dispose"},{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:15 HOST1 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:14 HOST1 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:09 HOST1 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 16:08 HOST1 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 15:58 HOST1 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
05/11/2014 15:57 HOST1 {"Name":"AccessViolationException","Method":"ComputeKeys","Source":"System.Core","Message":"Attempted to read or write protected memory. This is often an indication that other memory is corrupt.","StackTrace":{"Lines":[{"LineNumber":0,"Method":"ComputeKeys"},{"LineNumber":0,"Method":"Sort"},{"LineNumber":0,"Method":"MoveNext"},{"LineNumber":0,"Method":".ctor"},{"LineNumber":0,"Method":"ToList"},{"LineNumber":0,"Method":"lambda_method"},{"LineNumber":0,"Method":"Execute"}]}}
As you can see, it occurred at about the same time ON BOTH HOSTS. Hence me wondering if this is not a .NET/programming issue - but a hardware one. Note they are both VMs and apparently build in the same manner. Both processes on the separate hosts died a death (we load-balance between the hosts).
Repository Refreshes are done using a System.Threading.Timer. However, looking at my repository logs, no repository refreshes LESS than an hour, and the longest repository to refresh, is 11 minutes - so there is no overlap.
Access to the data is protected using a ReadWriterLockSlim class. I know this is a bit obscure, and if anyone needs any additional information I'll be happy to provide what I can.
