94

I'm trying to assert the equality of two System.Drawing.Size structures, and I'm getting a format exception instead of the expected assert failure.

[TestMethod]
public void AssertStructs()
{
    var struct1 = new Size(0, 0);
    var struct2 = new Size(1, 1);

    //This throws a format exception, "System.FormatException: Input string was not in a correct format."
    Assert.AreEqual(struct1, struct2, "Failed. Expected {0}, actually it is {1}", struct1, struct2); 

    //This assert fails properly, "Failed. Expected {Width=0, Height=0}, actually it is {Width=1, Height=1}".
    Assert.AreEqual(struct1, struct2, "Failed. Expected " + struct1 + ", actually it is " + struct2); 
}

Is this intended behavior? Am I doing something wrong here?

Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Kyle
  • 3,170
  • 1
  • 17
  • 20
  • have you tried having `Assert.AreEqual(struct1, struct2, string.Format("Failed expected {0} actually is {1}`, struct1.ToString(), struct2.ToString()))`? – DiskJunky Feb 19 '13 at 19:03
  • That works fine; however I'm curious as to why Assert.AreEqual() can't format a string with structure types. – Kyle Feb 19 '13 at 19:08
  • @Kyle Out of curiosity, this isn't with the Silverlight compatible version of the Unit Testing framework, is it? I can reproduce it with those DLLs (haven't tried the full .NET framework version yet) EDIT: nevermind, tested with the full ones too and still failed. :) – Chris Sinclair Feb 19 '13 at 19:16
  • @ChrisSinclair nope, this is using whatever version of mstest that comes with Visual Studio 2010 ultimate. The test project itself is targeting .NET Framework 4 – Kyle Feb 19 '13 at 19:20
  • 4
    Not sure if you give a damn, but this works fine in NUnit. I have seen more "issues" like these in MStest. NUnit seems a bit more mature (to me at least). +1 for the post – bas Feb 19 '13 at 20:47
  • Wow, I ran into this bug a few years ago... It's still not fixed? :( – Thorarin Feb 20 '13 at 13:03

4 Answers4

100

I've got it. And yes, it's a bug.

The problem is that there are two levels of string.Format going on here.

The first level of formatting is something like:

string template  = string.Format("Expected: {0}; Actual: {1}; Message: {2}",
                                 expected, actual, message);

Then we use string.Format with the parameters you've supplied:

string finalMessage = string.Format(template, parameters);

(Obviously there's cultures being provided, and some sort of sanitization... but not enough.)

That looks fine - unless the expected and actual values themselves end up with braces in, after being converted to a string - which they do for Size. For example, your first size ends up being converted to:

{Width=0, Height=0}

So the second level of formatting is something like:

string.Format("Expected: {Width=0, Height=0}; Actual: {Width=1, Height=1 }; " +
              "Message = Failed expected {0} actually is {1}", struct1, struct2);

... and that's what's failing. Ouch.

Indeed, we can prove this really easily by fooling the formatting to use our parameters for the expected and actual parts:

var x = "{0}";
var y = "{1}";
Assert.AreEqual<object>(x, y, "What a surprise!", "foo", "bar");

The result is:

Assert.AreEqual failed. Expected:<foo>. Actual:<bar>. What a surprise!

Clearly broken, as we weren't expecting foo nor was the actual value bar!

Basically this is like a SQL injection attack, but in the rather less scary context of string.Format.

As a workaround, you can use string.Formatas StriplingWarrior suggests. That avoids the second level of formatting being performed on the result of formatting with the actual/expected values.

BlueRaja - Danny Pflughoeft
  • 84,206
  • 33
  • 197
  • 283
Jon Skeet
  • 1,421,763
  • 867
  • 9,128
  • 9,194
  • Thanks for the detailed answer Jon! I did end up using StriplingWarriors work around. – Kyle Feb 19 '13 at 20:49
  • Has anyone submitted a bug report for this? – Kevin Feb 20 '13 at 06:27
  • @Kevin: Yup - although internally, so I'm not sure whether progress will be publicly visible until it's fixed. – Jon Skeet Feb 20 '13 at 06:48
  • 1
    @Kevin I put one in to MS as well once it was confirmed a bug. http://connect.microsoft.com/VisualStudio/feedback/details/779528/assert-areequal-t-method-t-t-string-object-throws-a-format-exception-when-comparing-structures if you want to track it publicly. – Kyle Feb 20 '13 at 13:40
43

I think you've found a bug.

This works (throws an assert exception):

var a = 1;
var b = 2;
Assert.AreEqual(a, b, "Not equal {0} {1}", a, b);

And this works (outputs the message):

var a = new{c=1};
var b = new{c=2};
Console.WriteLine(string.Format("Not equal {0} {1}", a, b));

But this doesn't work (throws a FormatException):

var a = new{c=1};
var b = new{c=2};
Assert.AreEqual(a, b, "Not equal {0} {1}", a, b);

I can't think of any reason this would be expected behavior. I'd submit a bug report. In the meantime, here's a workaround:

var a = new{c=1};
var b = new{c=2};
Assert.AreEqual(a, b, string.Format("Not equal {0} {1}", a, b));
StriplingWarrior
  • 151,543
  • 27
  • 246
  • 315
5

I agree with @StriplingWarrior that this does indeed appear to be a bug with the Assert.AreEqual() method on at least 2 overloads. As StiplingWarrior has already pointed out, the following fails;

var a = new { c = 1 };
var b = new { c = 2 };
Assert.AreEqual(a, b, "Not equal {0} {1}", a, b);

I've been doing a little experimenting on this further on being a little more explicit in code usage. The following doesn't work either;

// specify variable data type rather than "var"...no effect, still fails
Size a = new Size(0, 0);
Size b = new Size(1, 1);
Assert.AreEqual(a, b, "Not equal {0} {1}", a, b);

And

// specify variable data type and name the type on the generic overload of AreEqual()...no effect, still fails
Size a = new Size(0, 0);
Size b = new Size(1, 1);
Assert.AreEqual<Size>(a, b, "Not equal {0} {1}", a, b);

This got me thinking. System.Drawing.Size is a struct. What about objects? The param list does specify that the list after the string message is params object[]. Technically, yes structs are objects...but special kinds of objects, ie, value types. I think this is where the bug lies. If we use our own object with a similar usage and structure to Size, the following actually does work;

private class MyClass
{
    public MyClass(int width, int height)
        : base()
    { Width = width; Height = height; }

    public int Width { get; set; }
    public int Height { get; set; }
}

[TestMethod]
public void TestMethod1()
{
    var test1 = new MyClass(0, 0);
    var test2 = new MyClass(1, 1);
    Assert.AreEqual(test1, test2, "Show me A [{0}] and B [{1}]", test1, test2);
}
DiskJunky
  • 4,750
  • 3
  • 37
  • 66
  • 1
    The problem is not whether it is `class` or `struct`, but whether the `ToString` value contains curly brackets that look like a `String.Format`. – Jean Hominal Mar 22 '13 at 13:02
3

I think the first assert is incorrect.

Use this instead:

Assert.AreEqual(struct1, 
                struct2, 
                string.Format("Failed expected {0} actually is {1}", struct1, struct2));
Peter Mortensen
  • 30,738
  • 21
  • 105
  • 131
Polaris
  • 3,643
  • 10
  • 50
  • 61
  • According to the documentation I should be able to call AreEqual with a formatted string. http://msdn.microsoft.com/en-us/library/ms243436%28v=vs.100%29.aspx , specifically parameters Type: System.Object[] An array of parameters to use when formatting message. – Kyle Feb 19 '13 at 19:24