0

We're experiencing an issue with ASP.Net runtime compilation.

At the start (and after restarts) of the IIS, user controls and layouts compile just fine without any problem. But at some point during the lifecycle, the runtime compilation simply stops working. A restart of the IIS process makes it work again.

After searching through a myriad of different posts, we've done some extra debugging, but we are still stumped as to what causes our problem.

To keep this to the point, I'll skip explaining all tests we have done, and jump right to what I believe is closest to the heart of the error.

We have switched our compilation.tempDirectory to a custom folder used exclusively by this web application, and we have set a procmon to see all file changes in this folder. Once the error starts occuring we can see that the temporary files are actually written to this drive, and csc.exe is started (and during this, conhost.exe, CcmExec.exe and many other processes), we see no errors in procmon, but after csc.exe has run, it deletes the temporary files without leaving the compiled versions. (and the asp.net error screen shows us csc.exe failed, but not exactly what failed). The temporary files are all created (except for the resulting dll) - .0.cs, .1.cs, .tmp, .cmdline, .out, .err are all there for a short time. But immediatly after creation, they're all deleted again, and the application cannot find them.

Does anyone have any clue as to what causes this to fail after the process has run for a while? The exact same file compiles just fine if we run it just after an IIS restart, but after a while it seems that something during the runtime compilation fails, causing the ASP/w3wp/csc process to delete all signs of the temporary files, not creating the dll, and making the functionality fail.

morten.engel
  • 101
  • 4
  • Might be worth noting that this server has run for well over a year, and this problem only arose one month ago. While it's not guaranteed to happen daily, it often happens multiple times a day, requiring us to restart. This server is a CM server (only used by content editors), the exact same code runs on our CD server which is way more busy and has not shown signs of this error. However our CM server would be more prone to the error, as this has alot more usercontrols which are run much more rarely, so the chance they're compiled early in the application lifecycle is lesser. – morten.engel Jun 16 '16 at 12:30

1 Answers1

0

Finally figured out what caused my problem.

We had an external module installed, which ran another third party exe file to do some optimizations. Running external programs, starts a conhost.exe process under the application pool identity.

The second third party (the exe) could under some circumstances fail due to file locks, which caused the exe as well as the conhost.exe to hang. Once enough conhost.exe's were running, the application pool were no longer allowed to start new ones to run the runtime compilation. This caused the runtime compilation to fail, and the web application to automatically clean up the temporary files.

morten.engel
  • 101
  • 4