2

(https://benchmarkdotnet.org/)

Is it possible to skip a single benchmark section for a specific runtime?

For example i want to test several functions for 4.7.2 and Core 3.1, but one should only be benched on Core31

[Benchmark]
public void CoreOnly()
{
#if NETCOREAPP3_1
    //some stuff i only want to test with 3.1
#endif
}

[Benchmark]
public void General()
{
    //some stuff i want to test on all runtimes
}

Thats how i have done this until now. Is there a better way?

Crimson
  • 103
  • 6
  • How about moving the `#if` and `#endif` part outside the method to encapsulate it altogether? – Lasse V. Karlsen Dec 20 '19 at 08:53
  • then it wont detect the CoreOnly() benchmark, and only General is tested. And on my version, CoreOnly() is added on 4.7.2-report with super fast times XD. i want CoreOnly() to be hidden on the result-report (an no i dont want to delete it from the report manually ;) ) – Crimson Dec 20 '19 at 08:58

1 Answers1

1

This is impossible by design.

When running the host process targetting XYZ framework, BDN is using the reflection to obtain the list of available methods (benchmarks). If you are using #if defines, then the list of benchmarks is going to be different per host process target framework.

Performance repo docs describe how to compare multiple runtimes performance here: https://github.com/dotnet/performance/blob/master/docs/benchmarkdotnet.md#multiple-runtimes

The host process needs to be the lowest common API denominator of the runtimes you want to compare!

You can read more about testing multiple runtimes here: https://benchmarkdotnet.org/articles/configs/toolchains.html#multiple-frameworks-support

Adam Sitnik
  • 1,256
  • 11
  • 15