mod_mono: Random CompilationException errors

Question

After a server restart I sometimes get random CompilationException errors like this:

CompilationException: CS0006: Metadata file `/tmp/apache-temp-aspnet-0/ca373c84/assembly/shadow/10cad1cb/02cb7ade_1201ab15_00000001/ZedGraph.dll' could not be found -> HttpException: Single file build failed

and you can't access the application. If I restart the server again then the problem disappears.

The file that triggers the error can be one of my own dll's or a dependency, like mysql.dll for instance.

I am using mono 2.10.2, CentOS release 5.7 (Final), Apache and mod_mono on a Linode VPS.

Any idea of what can be the cause?

Edit: Since my troubles started when I moved to Linode and they use XEN, I have found the option --with-xen_opt=yes that looks promissing: http://mono-project.com/Advanced_Mono_Compile_Options

When you say randomly, does your application randomly restart, causing this compilation error? Or initial compilation errors when mod_mono starts your application? — Patrick McEvoy, Jan 23 '12 at 13:48
mod_mono autorestarts the application after n requests I think and I also have a cron task that autorestarts apache when it is not responding. — Santiago Corredoira, Jan 24 '12 at 10:38
I will as soon as they deploy the packages because I want to keep the official yum install. — Santiago Corredoira, Jan 30 '12 at 12:02
I updated to .net4 from .net2 but I still get the random errors. — Santiago Corredoira, Feb 01 '12 at 15:54

score 2 · Accepted Answer · answered Jan 23 '12 at 14:05

Sounds like a bug (race condition?) with the shadow copying and compilation. There have been others found and fixed before... see http://grendello.blogspot.com/2007/08/new-modmono-and-xsp-developments.html

And this is a report of a similar problem in 2006 http://lists.ximian.com/pipermail/mono-list/2006-March/030970.html

Rather than rebooting the system, you can try restarting apache. Note, I have found it can help to stop apache and then start, rather than a restart.

You can also enable the mod_mono control panel and restart the application from there. Add this in apache config.

<Location /monocp>
  SetHandler mono-ctrl
  Order deny,allow
  Deny from all
  Allow from 127.0.0.1 <YOURIP>
</Location>

To automate this process take a look at Monit.

thats what I do now, I have created a cron task every minute that checks it is running and if not, then restart apache. The problem is that sometimes only a single part of the web application fails and the rest work fine. — Santiago Corredoira, Jan 24 '12 at 10:37

score 0 · Answer 2 · edited May 23 '17 at 11:55

0

FWIW, I had the exact same issue on a dedicated server with no virtualization (8 core i7, x86 Ubuntu Server Edition), so I do not believe this to be a Xen-induced issue. For either the OP (SCL) or anyone else with this issue, I have documented my work around in another SO post here, as well as having filed a bug report with Mono/Xamarin.

It does seem to be a race condition, but not related to virtualization.

edited May 23 '17 at 11:55

Community

1
1

answered Apr 25 '12 at 05:06

Mahmoud Al-Qudsi

28,357
12
85
125

Very interesting. Since I moved to dedicated servers I haven't seen the problem any more, but since you are having them, I am going to change the accepted answer. Thank you for the info. – Santiago Corredoira Apr 25 '12 at 11:19
Race conditions can be more likely to happen on systems under more load. Since a VPS is sharing resources that could explain why it appears to be a virtualisation problem but that others with dedicated hardware (probably under heavy load) still experience the issue. – Patrick McEvoy Jun 17 '12 at 17:55

Santiago Corredoira · Answer 3 · 2012-04-25T11:21:39.117

0

It seems that the problems where caused by XEN virtualization and mono precompiled binaries. I compiled mono with the option --with-xen_opt=yes but I wasn't able to compile XSP so I couldn't test if that solved it for sure but I moved exactly the same application to a dedicated server and after 2 million requests I didn't experience any error.

Update: Check Mahmoud answer

edited Apr 25 '12 at 11:21

answered Feb 09 '12 at 15:10

Santiago Corredoira

47,267
10
52
56

mod_mono: Random CompilationException errors

3 Answers3

Linked