BenchmarkDotNet StringConcats vs StringFormat

Question

I'm using .net core 2.0.5 on windows 10 64bit OS. Everytime, I will concate string by using string.Concat or string.Format because I learnt using string "abc" + "def" is the worse by performance. ( Memory usage is a different topic. ) - I recently benchmarked string concates but the results are quite interesting;


using System;

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Attributes.Columns;
using BenchmarkDotNet.Attributes.Exporters;

namespace TryBenchmark.ConsoleApp.Benchmarks
{
    [RPlotExporter, RankColumn]
    public class StringConcatVsStringFormat
    {
        [Benchmark]
        public void StringConcat001()
        {
            string result = "test1 " + "test2 " + "test3";

            if (result != "test1 test2 test3")
            {
                throw new InvalidOperationException("Tests are faulty !");
            }
        }

        [Benchmark]
        public void StringConcat002()
        {
            string result = String.Concat("test1 ", "test2 ", "test3");

            if (result != "test1 test2 test3")
            {
                throw new InvalidOperationException("Tests are faulty !");
            }
        }

        [Benchmark]
        public void StringConcat003()
        {
            string name1 = "test1";
            string name2 = "test2";
            string name3 = "test3";

            string result = $"{name1} {name2} {name3}";

            if (result != "test1 test2 test3")
            {
                throw new InvalidOperationException("Tests are faulty !");
            }
        }

        [Benchmark]
        public void StringFormat()
        {
            string result = String.Format("{0} {1} {2}", "test1", "test2", "test3");

            if (result != "test1 test2 test3")
            {
                throw new InvalidOperationException("Tests are faulty !");
            }
        }
    }
}

Results;


StringConcatVsStringFormat.StringFormat: DefaultJob
Runtime = .NET Core 2.0.5 (CoreCLR 4.6.26020.03, CoreFX 4.6.26018.01), 64bit RyuJIT; GC = Concurrent Workstation
Mean = 149.2378 ns, StdErr = 0.9310 ns (0.62%); N = 79, StdDev = 8.2749 ns
Min = 140.0365 ns, Q1 = 143.8430 ns, Median = 145.9400 ns, Q3 = 150.4091 ns, Max = 172.1811 ns
IQR = 6.5661 ns, LowerFence = 133.9939 ns, UpperFence = 160.2582 ns
ConfidenceInterval = [146.0541 ns; 152.4215 ns] (CI 99.9%), Margin = 3.1837 ns (2.13% of Mean)
Skewness = 1.29, Kurtosis = 3.57, MValue = 2
-------------------- Histogram --------------------
[139.525 ns ; 142.434 ns) | @@@@@@@@@@@
[142.434 ns ; 145.809 ns) | @@@@@@@@@@@@@@@@@@@@@@@@@@@@
[145.809 ns ; 150.658 ns) | @@@@@@@@@@@@@@@@@@@@@@
[150.658 ns ; 156.460 ns) | @@@@
[156.460 ns ; 160.840 ns) | @
[160.840 ns ; 164.215 ns) | @@@@@@@
[164.215 ns ; 169.531 ns) | @@@@
[169.531 ns ; 173.869 ns) | @@
---------------------------------------------------

Total time: 00:03:26 (206.65 sec)

// * Summary *

BenchmarkDotNet=v0.10.14, OS=Windows 10.0.16299.248 (1709/FallCreatorsUpdate/Redstone3)
Intel Core i7-7500U CPU 2.70GHz (Kaby Lake), 1 CPU, 4 logical and 2 physical cores
Frequency=2835939 Hz, Resolution=352.6169 ns, Timer=TSC
.NET Core SDK=2.1.100
  [Host]     : .NET Core 2.0.5 (CoreCLR 4.6.26020.03, CoreFX 4.6.26018.01), 64bit RyuJIT  [AttachedDebugger]
  DefaultJob : .NET Core 2.0.5 (CoreCLR 4.6.26020.03, CoreFX 4.6.26018.01), 64bit RyuJIT


          Method |       Mean |     Error |    StdDev |      Median | Rank |
---------------- |-----------:|----------:|----------:|------------:|-----:|
 StringConcat001 |   1.043 ns | 0.0608 ns | 0.1558 ns |   0.9972 ns |    1 |
 StringConcat002 |  26.680 ns | 0.5303 ns | 0.5445 ns |  26.7079 ns |    2 |
 StringConcat003 | 143.028 ns | 2.4180 ns | 2.2618 ns | 143.9356 ns |    3 |
    StringFormat | 149.238 ns | 3.1837 ns | 8.2749 ns | 145.9400 ns |    4 |

// * Warnings *
Environment
  Summary -> Benchmark was executed with attached debugger

How StringConcat001 is the fastest?
Am I doing something wrong? Or I mis-configurated benchmarkdotnet?

You are probably getting hit by dead code elimination. Look at http://benchmarkdotnet.org/Guides/GoodPractices.htm#avoid-dead-code-elimination. Make your methods return the value computed in the method. Get rid of the exception tests too, they are just confusing in a micro benchmark. — omajid, Apr 09 '18 at 14:30
This was micro-optimized in the C# compiler itself. Written to transform a sequence of operator+() calls to the corresponding String.Concat() overload. Since you are using string literals, it even knows how to concatenate at compile-time. Pretty visible when you use ildasm.exe to look at the generated msil. Eric Lippert worked on this btw. — Hans Passant, Apr 09 '18 at 14:35
@omajid Thanks for comments but I see similar results if I return string and remove throw exceptions. — Lost_In_Library, Apr 10 '18 at 06:02

score 4 · Accepted Answer · answered Apr 10 '18 at 10:36

The C# compiler is smart enough to detect that the value of string result = "test1 " + "test2 " + "test3"; is a constant.

You can use ILSpy to see what the compiler produces:

[Benchmark]
public void StringConcat001()
{
    if (!("test1 test2 test3" != "test1 test2 test3"))
    {
        return;
    }
    throw new InvalidOperationException("Tests are faulty !");
}

To trick the compiler you either have to put these values to a public, non-readonly fields or use them as arguments of given method. BenchmarkDotNet 0.10.14 allows you to provide values as arguments.

[RPlotExporter, RankColumn]
[MarkdownExporterAttribute.StackOverflow]
[MemoryDiagnoser]
public class StringConcatVsStringFormat
{
    [Benchmark]
    [Arguments("test1 ", "test2 ", "test3")]
    public string StringConcat001(string arg1, string arg2, string arg3)
        => arg1 + arg2 + arg3;

    [Benchmark]
    [Arguments("test1 ", "test2 ", "test3")]
    public string StringConcat002(string arg1, string arg2, string arg3)
        => String.Concat(arg1, arg2, arg3);

    [Benchmark]
    [Arguments("test1 ", "test2 ", "test3")]
    public string StringConcat003(string arg1, string arg2, string arg3)
        => $"{arg1} {arg2} {arg3}";

    [Benchmark]
    [Arguments("test1 ", "test2 ", "test3")]
    public string StringFormat(string arg1, string arg2, string arg3)
        => string.Format("{0} {1} {2}", arg1, arg2, arg3);
}

Gives following results on my PC:

BenchmarkDotNet=v0.10.14, OS=Windows 10.0.16299.309 (1709/FallCreatorsUpdate/Redstone3)
Intel Xeon CPU E5-1650 v4 3.60GHz, 1 CPU, 12 logical and 6 physical cores
Frequency=3507503 Hz, Resolution=285.1031 ns, Timer=TSC
  [Host]     : .NET Framework 4.7.1 (CLR 4.0.30319.42000), 32bit LegacyJIT-v4.7.2633.0
  DefaultJob : .NET Framework 4.7.1 (CLR 4.0.30319.42000), 32bit LegacyJIT-v4.7.2633.0


          Method |   arg1 |   arg2 |  arg3 |      Mean |     Error |    StdDev | Rank |  Gen 0 | Allocated |
---------------- |------- |------- |------ |----------:|----------:|----------:|-----:|-------:|----------:|
 StringConcat001 | test1  | test2  | test3 |  25.95 ns | 0.1755 ns | 0.1642 ns |    2 | 0.0091 |      48 B |
 StringConcat002 | test1  | test2  | test3 |  25.66 ns | 0.3480 ns | 0.3085 ns |    1 | 0.0091 |      48 B |
 StringConcat003 | test1  | test2  | test3 | 112.32 ns | 0.9539 ns | 0.8923 ns |    4 | 0.0098 |      52 B |
    StringFormat | test1  | test2  | test3 | 111.62 ns | 0.9982 ns | 0.8849 ns |    3 | 0.0098 |      52 B |

Thanks for your answer. It's very interesting that compiler sees const value and optimize it. Very good. I learnt something new, thanks. But still, you got StringConcat001 is the fastests ( almost ). But how it's possible? I beleive concating strings with "+" sign should be slowest. Because ".Concat()" and ".Format()" methods and has been written for this purpose (concating). Am I wrong? — Lost_In_Library, Apr 10 '18 at 12:47
`String.Format` has to parse the string and then concatenate it. This is why concat is faster. If you compare the IL code you will see that the IL for `StringConcat001` and `StringConcat002` is identical. The same goes for `StringConcat003` and `StringFormat`. — Adam Sitnik, Apr 10 '18 at 17:54
You should try to compare concat vs string builder vs cached string builder. Sample code for string builder cache can be found [here](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/src/System/Text/StringBuilderCache.cs). If you want to learn more about string manipulation you should check [string.manipulation.cs](https://github.com/dotnet/coreclr/blob/master/src/mscorlib/shared/System/String.Manipulation.cs) from the CoreCLR repo. — Adam Sitnik, Apr 10 '18 at 17:57

BenchmarkDotNet StringConcats vs StringFormat

1 Answers1