39

I am surprised to observe that mono is faster than .NET. Does anyone know why is it so? I was expecting mono to be slower than .NET but wasnt the case atleast with my experiments.

I have a windows xp laptop with .NET framework. I am running CentOS on vmware vmplayer on top of windows xp. I wanted to try mono. So grabbed the Mono 2.6.1 sources and installed it on CentOS in vmplayer. I have writen a test webservice application using .Net 2.0 executed it on wndows, it worked, I transferred the binary to the centos in the vmplayer without any recompilation and executed it on centos. Hurray it worked! Life is good but then something else lured my attention. The execution of the application on centos seemed to be faster. I did not believe my eyes.

To confirm my observation, I eliminated network out of the equation because network reponse depends on lot of external factors.

I grabbed small dummy loop code from internet, compiled it in visual studio executed in windows as well as CentOS, the results are as follows

Output on windows console is HelloConsole\bin\Debug>HelloConsole.exe
Result =2.66666664666712E+24
37443.6077769661 ms


Output on Centos console is [rupert@bagvapp Math.Pow]$ mono HelloConsole.exe
Result =2.66666664666712E+24
28790.6286 ms

If anyone can explain this behavior,that would be great. my layman's understanding is Mono's implementation is more efficient than .NET framework. Even if I assume Mono's Math implementation is only efficient. But lot of implementations like processing financial data, graphics calculations depend lot on Math library. It would be more interesting to perform the test on Mono/Centos directly without vmware but that needs some time. I will give it a try may be next weekend. 

public static void DummyLoop()
    {
        double sumOfPowers = 0;
        int count = Convert.ToInt32(ConfigurationManager.AppSettings["count"]);

            for (int i = 0; i < count; i++)
            {
                sumOfPowers += Math.Pow(i, 2);   
            }


        Console.WriteLine("Result =" + sumOfPowers);   
    }


    static void Main(string[] args)
    {
        Stopwatch stopWatch = new Stopwatch();
        stopWatch.Start();
        DummyLoop();
        stopWatch.Stop();
        double ms = (stopWatch.ElapsedTicks * 1000.0) / Stopwatch.Frequency;
        Console.WriteLine(string.Concat(ms.ToString(), " ms"));
        Console.ReadLine();
    }
funwithcoding
  • 2,412
  • 2
  • 30
  • 41
  • 3
    How unusual. http://flyingfrogblog.blogspot.com/2009/01/mono-22.html – Hans Passant Mar 07 '10 at 20:08
  • Hi Ben M, Convert.ToInt32 and ConfigurationManager.AppSettings are not in loop. So it would be only a increment.Any better ideas to benchmark than a loop? – funwithcoding Mar 07 '10 at 20:12
  • @nobugz, thanks for the article, however I am using Mono 2.6.1 which is more recent than the article. – funwithcoding Mar 07 '10 at 20:13
  • @funwithcoding, my bad. And yeah, the benchmark in the article linked to by nobugz would be a start. Since the code is freely available, you could probably try it yourself. – Ben M Mar 07 '10 at 20:17
  • 2
    While this specific case is interesting ,you should also benchmark something more complicated. Most of our programs run about 2-3 times slower on mono, IO not included. – nos Mar 07 '10 at 20:20
  • Did you run the windows code from Visual Studio or outside of the IDE? – Ryan Alford Mar 07 '10 at 20:44
  • @Eclipsed4utoo Outside the IDE from the console. – funwithcoding Mar 07 '10 at 20:49
  • @Ben, That benchmark library needs some time to convert to C#, so I have created a lame implementation. Still Mono/CentOS/Vmware/Windows beats .NET/Windows. – funwithcoding Mar 07 '10 at 21:06
  • @nos What type of application is yours? Is it GUI application or console application? – funwithcoding Mar 07 '10 at 22:15
  • @funwithcodin It is message processing/decoding software, not GUI – nos Mar 08 '10 at 08:13
  • @nos IO seems to fairly similar on mono as well as .NET, string handling is faster in .NET, Math is faster in Mono. What are your experiences with IO & string handling? – funwithcoding Mar 08 '10 at 12:51
  • 2
    You're not really benchmarking Mono with this test - essentially, you're benchmarking the implementation of one particular mathematical function, `Math.pow()`, and that's not likely to reflect the overall performance of the framework and VM. If you were to simplify the calculation inside the loop, e.g. something trivial like `sumOfPowers += 1`, you'd be benchmarking the for-loop and a trivial variable-assignment, which tells you something, but still isn't a benchmark of Mono, as you're also (hugely) benchmarking the performance of the C# compiler implementation. Many factors... – mindplay.dk Sep 03 '13 at 12:33

4 Answers4

59

This was a discussion about this on the Mono mailing list not too long ago. The reasoning they provided is Mono is extremely optimized under Linux (exactly how did not seem to be mentioned). But Mono is not the only variable here, if you are running Mono on Linux. It's a different OS, different process schedulers and various kernel level subsystems can have a markedly different effects on performance, independent of the Mono team's work. Mono also makes heavy use of compiler intrinsics, which .NET may or may not do.

However, most of the time on a Mono on Windows vs .NET on Windows comparison, you might find .NET beating Mono more often then not. But even then, Mono might not always lose. In fact, for code specifically optimized for Mono (eg: using Mono.SIMD), you might get an order of magnitude better performance on Mono vs .NET, regardless of platform.

joemoe
  • 5,734
  • 10
  • 43
  • 60
  • 1
    @joemoe Thanks for the clarification. I will try other comparisons like I/O, database. Seems like Mono is optimized for Linux and .NET is optimised for windows. As Linux beats windows in performance in many ways it makes sense that Mono on Linux beats .NET on windows. – funwithcoding Mar 07 '10 at 21:54
  • 2
    Quote from mono mailing list "To clear things up, the IL code that Mono generates from the code sources is the same for all OSes, but the native code generated (JITed) from the IL code is what varies depending on the OS. And the generated native code is more or less efficient depending on the OS." – funwithcoding Mar 07 '10 at 22:21
  • 5
    another thing to note is that any operation which requires lots of memory allocation and garbage collection is slow on mono, they are however working on speeding this us using a new garbage collector. – trampster Mar 07 '10 at 23:08
  • 1
    The current garbage collector chokes in 2 major ways: (1) collection performance degrades linearly with # of objects on the heap. (2) excessive allocations/deallocations might lead to memory fragmentation. The new garbage collector mostly eliminates these problems. – joemoe Mar 08 '10 at 00:51
  • Re:collection performance degrades linearly with # of objects on the heap, I think you'd be hard-pressed to find a GC algorithm that wasn't linear in the # objects. The question is how low you can make the linear scaling factor on your GC algorithm. – naasking Jun 10 '10 at 01:26
  • @naasking Reference counting with a trial-deletion cycle finder? – J D Feb 01 '13 at 20:30
  • Indeed, ref counting is linear in the number of deleted objects, where copy collection is linear in the number of live objects. So assuming the generational hypothesis is true, ref counting tends to be good for the old generation where objects die slowly, and copy collection tends to be good for the nursery where most objects die young. This is known as "age-oriented" collection. – naasking Feb 05 '13 at 02:37
  • @naasking - I think you err in saying "ref counting is linear in ..". IMHO, ref counting deletion is order ZERO for all objects that are not in a cycle. Or to say it another way, is there any work that is done during deletion of an object, that is not also done in ANY GC mechanism? AFAIK, once you've identified a deleted object, its the same, so cancels out. The primary cost of ref counting is in UPDATING the reference COUNT, as each REFERENCE (not each object) begins and ends its lifetime. E.g. parameters passed as a ref, at beginning and end of method scope. – ToolmakerSteve Mar 02 '17 at 04:49
  • @naasking - sorry, I can't let this go till I've made the key point clear: copy collection is linear in the number of objects to be scanned - the number of objects in the generation - REGARDLESS of how many live or die. If all objects in generation are dying, ref count is also linear with size of generation. EXACTLY THE SAME LINEARITY as copy collection, in that EXTREME case. IN ALL OTHER CASES, ref counting has better linearity. Ref counting would ALWAYS win, if not for UPDATE cost!! (and the need to eventually run some other collection, to find loops) – ToolmakerSteve Mar 02 '17 at 05:37
  • @ToolmakerSteve I was talking about storage reclamation costs, which are exactly like I said. – naasking Mar 02 '17 at 18:34
  • @naasking - What you said sounded like ref counting DELETION was MORE EXPENSIVE than sweep/copying, for a young generation where most objects are dying. That's only true if you exclude the SCANNING cost of sweep/copy, which makes no sense to do - that's ignoring virtually all the expense; it is meaningless. The SIGNIFICANT extra cost of ref counts comes earlier, as UPDATE ref counts, NOT during deletion. Because of this, your description of "age-oriented" collection is inaccurate. The reason to use ref counting in old generation, and sweep/copy in young, is NOT what you say it is. – ToolmakerSteve Mar 03 '17 at 00:09
  • It is more expensive. Copying GC only touches live objects and the whole generation can be freed at once. Ref counting touches only *possibly dead* objects, and utilizes a free-list style allocator, so there's a per-object deallocation cost that simply doesn't exist with copying GC, hence linear in the number of dead objects. If your gen consists of mostly dead objects, ref counting is obviously way more expensive. Which doesn't even touch on the cost of ref counting's incremental count updates. Ref counting also needs local tracing to collect cycles, so you're not even saving tracing costs. – naasking Mar 04 '17 at 13:41
10

You're not really making much of a comparison here. You are basically comparing one function (Math.Pow) between the two. Presumably a real application will do more things than this one function.

Mono's Math.Pow looks to be optimized by implementing this in C. I do not know how .Net implements it. It may be implemented fully in managed code.

Likely you will find that both runtimes are fast enough for every day needs.

jpobst
  • 9,982
  • 1
  • 29
  • 33
  • @jpobst Thats the point I want to make, Not only Math.Pow, many math functions seems to be faster in Mono. Imagine an financial application which does lot of math operations or scientific applications,for them probably Mono serves better than .NET and that to with ease of programming. I completely agree that both runtimes are fast enough for every day needs. In fact for the most of Line of business applications, we dont care performance as much as we do development time, otherwise we wouldnt be using C# in first place, If it is only for performance we would have used C++, C or assembly. – funwithcoding Mar 07 '10 at 22:37
  • I wanted to understand why it is faster in Mono when both share the same IL. Now I realize that though the IL is same, OS is different and the way IL is converted to native code is different. – funwithcoding Mar 07 '10 at 22:41
-3

All this really tests is the implementation of the timer - which are typically bad tests because no two OS implement them the same. Some use a hard timer which is very exact (Linux) and some have a timer than can be preempted (Windows) based on system needs. If you want to benchmark a call you have avoid intermediate calls or you are probably timing them not what you are thinking you are testing. Math POW is going to be nearly the same on both platforms since the math to calculate it hasn't changed in a hundred years. I learn toward OS differences with the timer more than anything.

Mindmaster
  • 11
-19

Well, I'm not surprised it executes the same bytecode faster.

If you try the Windows version of Mono, I would expect a smaller performance difference, except in cases that involve SIMD optimized operations.

Sam
  • 7,252
  • 16
  • 46
  • 65
Marko
  • 30,263
  • 18
  • 74
  • 108