0

After moving a WCF service from one production server to another, where the configuration is very similar, custom event logging via Event Trace for Windows has stopped working, but just for one app.

The error is being thrown in the ctor of the EventProvider class and it is a Win32 "Not enough storage" error.

The WCF service is a 'concurrency mode multiple, instance context mode per call' thread per call configuration. At time of monitoring 60 threads belonged to the process. The EventProvider ctor is invoked per call. It is IIS/WAS hosted with AppFabric.

Another app on the same server is working OK.

I have no idea how to diagnose this. If anyone can even suggest a starting point I'd be grateful.

Sentinel
  • 3,582
  • 1
  • 30
  • 44

1 Answers1

1

OK this turned out to be related to VMWare configuration. The machine is a 12Gb server, but was configured to have 6Gb permanently reserved, with 6Gb taken from a pool. With a lot of memory pressure and swapping on the physical level random Win32 exceptions started to get thrown in the VM. The solution is to make more memory available.

UPDATE: The above was coincidence, it is not related to VMWare most likely.

The issue returned after a month. It seems that what something on the server has changed which slows down garbage collection, and my per-call wcf service is not disposing EtwRegistration handles explicitly (ie. I am not explicitly Disposing the EventProvider). Experiments show that there is a limit of 1000 EventProviders per process. The change in performance on the server resulted in a handle leak that hit that limit.

Further update: If anyone would like to increase the number of providers, instead of forcing cleanup for whatever reason, I think this might help http://support.microsoft.com/kb/2583244

Sentinel
  • 3,582
  • 1
  • 30
  • 44