2

I am trying to Create multiple assemblies at runtime. I am using Roslyn to compile each assembly in serial.

I am not able to leverage multithreading because Roslyn internally uses multithreading for compiling a single dll. The compilation time of a single dll with Roslyn is less than that of CodeDom but since with CodeDom I can parallel compile multiple dlls, the actual time for CodeDom is much less.

I tried to set the concurrent build option for Roslyn compilation to false and then used multithreading in my code, but this is also not efficient because the compile time for a single dll is more now. I am wondering if there is a work around for this issue

Update for timing analysis:

I have an 8 core processor. I have different sets of files on disk to be used for compiling 72 dlls.

In the first case, I am creating syntax trees from the files, creating references, creating options and compiling the dll using Emit for each dll in a serial fashion using a simple foreach loop. It takes a total time of 488589 ms

In the second case, I am using a parallel.ForEach with MaxDegreeOfParallelism = 7 and creating syntax trees, creating references, creating options and compiling the dll using Emit in the loop. It takes a total time of 514481 ms

In the both the case the concurrent build option is default: True.

As you can see there are no gains observed while using multithreading.

Anoop
  • 101
  • 1
  • 10
  • Its unclear how you are compiling the dlls but I assume it is be calling `Emit` on the `Compilation` object. If you have multiple compilation objects that you need to produce dlls ford you can parallelize this work using any .NET technique you like. What is preventing you from using `Parallel.Foreach` or some equivalent API? – Jonathon Marolf Feb 26 '18 at 16:23
  • I am using the Emit api. I tried using Parallel.Foreach but the compilation process is using multithreading internally as noted above..so it is resulting in more creation of more threads than cores(threads created by Parallel.ForEach + threads created by the EmitApi). So there is not much difference in compilation times whether I use Parallel.ForEach or not. – Anoop Feb 27 '18 at 08:39
  • Could you post your results? Most multi-threaded operations should use the shared threadpool so thrashing should non occur. What is the time for parallel vs serial execution? How many cores does this machine have? – Jonathon Marolf Feb 27 '18 at 20:59
  • @JonathonMarolf I have updated the description with the timing analysis. Please let me know if you need any more info. Thanks! – Anoop Mar 01 '18 at 12:48
  • take a look at the benchmarkd here: github.com/jmarolf/emit-benchmark sequential parallel emit calls are faster for me on a 4 core processor – Jonathon Marolf Mar 02 '18 at 16:49
  • @JonathonMarolf thanks for the benchmark. With the benchmark I too am seeing better times for parallel compilation(though the ratio is more like 1 : 0.5). But the syntax trees and code used for the dll are too simple. When the number of files(syntax trees) for the dll increases around(20) and as the complexity and size of code increases, we observed that the parallel compilation is not as efficient as obtained in the previous benchmark. – Anoop Mar 12 '18 at 13:42

0 Answers0