In .Net is System.Drawing.Image.Save deterministic?

Question

I'm trying to compare two Images via their byte content. However, they do not match.

Both images were generated from the same source image, using the same method with the same parameters. I am guessing that something in the image generation or the way I convert to a byte array is not deterministic. Does anyone know where the non-deterministic behavior is occurring and whether or not I can readily force deterministic behavior for my unit testing?

This method within my test class converts the image to a byte array - is image.Save deterministic? Is memStream.ToArray() deterministic?

private static byte[] ImageToByteArray(Image image)
{
    byte[] actualBytes;
    using (MemoryStream memStream = new MemoryStream())
    {
        image.Save(memStream, ImageFormat.Bmp);
        actualBytes = memStream.ToArray();
    }
    return actualBytes;
}

Here is the unit test, which is failing - TestImageLandscapeDesertResized_300_300 was generated from TestImageLandscapeDesert using the ImageHelper.ResizeImage(testImageLandscape, 300, 300) and then saving to a file before loading into the project's resource file. If all calls within my code were deterministic based upon my input parameters, this test should pass.

public void ResizeImage_Landscape_SmallerLandscape()
{
    Image testImageLandscape = Resources.TestImageLandscapeDesert;
    Image expectedImage = Resources.TestImageLandscapeDesertResized_300_300;
    byte[] expectedBytes = ImageToByteArray(expectedImage);
    byte[] actualBytes;
    using (Image resizedImage = ImageHelper.ResizeImage(testImageLandscape, 300, 300))
    {
        actualBytes = ImageToByteArray(resizedImage);
    }
    Assert.IsTrue(expectedBytes.SequenceEqual(actualBytes));
}

The method under test - this method will shrink the input image so its height and width are less than maxHeight and maxWidth, retaining the existing aspect ratio. Some of the graphics calls may be non-deterministic, I cannot tell from Microsoft's limited documentation.

public static Image ResizeImage(Image image, int maxWidth, int maxHeight)
{
    decimal width = image.Width;
    decimal height = image.Height;
    decimal newWidth;
    decimal newHeight;

    //Calculate new width and height
    if (width > maxWidth || height > maxHeight)
    {
        // need to preserve the original aspect ratio
        decimal originalAspectRatio = width / height;

        decimal widthReductionFactor = maxWidth / width;
        decimal heightReductionFactor = maxHeight / height;

        if (widthReductionFactor < heightReductionFactor)
        {
            newWidth = maxWidth;
            newHeight = newWidth / originalAspectRatio;
        }
        else
        {
            newHeight = maxHeight;
            newWidth = newHeight * originalAspectRatio;
        }
    }

    else
        //Return a copy of the image if smaller than allowed width and height
        return new Bitmap(image);

    //Resize image
    Bitmap bitmap = new Bitmap((int)newWidth, (int)newHeight, PixelFormat.Format48bppRgb);
    Graphics graphic = Graphics.FromImage(bitmap);
    graphic.InterpolationMode = InterpolationMode.HighQualityBicubic;
    graphic.DrawImage(image, 0, 0, (int)newWidth, (int)newHeight);
    graphic.Dispose();

    return bitmap;
}

You could extract the Bitmap/Graphics away from your class (With Adapters, for example), so you can test your actual logic and the calls made instead of the graphical components themselves (which you don't need to test - we already know Graphics work correctly). — Pierre-Luc Pineault, Jul 08 '15 at 19:27
The code has been used for quite some time, but we have not had much unit testing. I know it's poor practice, but I'm trying to rectify the situation by creating unit tests now. So, without changing the code under test, would you suggest I use MS Fakes to Shim the graphics calls and verify they are called with expected parameters. — Zarepheth, Jul 08 '15 at 19:31
One thing you can check is that does ImageToByteArray return the same thing when called multiple times on the same image. This is not the way way I've typically done this, which would be to use GetPixel on both and compare the values. It might be a little slower but then you can report exactly where the images differ. — Mike Zboray, Jul 08 '15 at 19:31
@mikez I called the `ImageToByteArray` method 5 times, 17 seconds apart, with the same source image. Comparing the first result against each of the subsequent results indicated that all 5 calls returned the same value. Non-deterministic behavior must be occuring within the method under test. — Zarepheth, Jul 08 '15 at 19:41
I noticed that you're using `PixelFormat.Format48bppRgb` to create your bitmap, but then you save it as a .bmp. That might be causing unintended behavior... the bit depth of your image will be reduced to 24bpp when you save it that way. Perhaps that affected how your reference image was saved? — RogerN, Jul 08 '15 at 20:15
The `Save()` method is deterministic. That is, it will always produce the same output given the same input. But the `Save()` method in different versions of .NET or the OS may produce different results, as would the method in a single environment if given different inputs. Please provide [a good, _minimal_, _complete_ code example](https://stackoverflow.com/help/mcve) that reliably reproduces the problem. — Peter Duniho, Jul 08 '15 at 20:38
Also, was your reference image (TestImageLandscapeDesertResized_300_300) created on the same machine? Since you're rescaling the bitmap, it might be important to know that bitmap stretching can be hardware-accelerated. Thus the exact output of calling `graphic.DrawImage` could vary depending on video card / drivers. — RogerN, Jul 08 '15 at 20:39
@RogerN at this time the reference image was created on the machine as the image being tested. However, I was expecting this to run correctly even on different machines; which means my assumption may fail when my team members run the test... — Zarepheth, Jul 08 '15 at 20:50
@PeterDuniho, so if the reference image in my resources is tested on a machine with a newer version of Windows, or the .Net version is incremented the test may fail - even if I get the unit test working on my machine? — Zarepheth, Jul 08 '15 at 20:52
Yes. There's no reason to expect the exact output to be identical. Note that I _would_ expect the actual _pixel_ data to be the same, at least for a non-lossy format like BMP (which is what you're saving here). But image files have other metadata that could vary according to implementation. And in your example, you appear to be scaling the image, which is also implementation-dependent. Again, with a good code example, you would probably get a specific answer. Without one, that's unlikely. — Peter Duniho, Jul 08 '15 at 21:13
By empirical testing I have discovered that this line `return new Bitmap(image);` returns an image with different properties than the original. The color palette is different and also the pixel format. — Zarepheth, Jul 08 '15 at 22:59

score 0 · Accepted Answer · answered Jul 08 '15 at 22:57

This eventually worked. I don't know whether or not this is a good idea for unit tests, but with the GDI+ logic being non-deterministic (or my logic interfacing with it), this seems the best approach.

I use MS Fakes Shimming feature to Shim the dependent calls and verify expected values are passed to the called methods. Then I call the native methods to get the required functionality for the rest of the method under test. And finally I verify a few attributes of the returned image.

Still, I would prefer to perform a straight comparison of expected output against actual output...

[TestMethod]
[TestCategory("ImageHelper")]
[TestCategory("ResizeImage")]
public void ResizeImage_LandscapeTooLarge_SmallerLandscape()
{
    Image testImageLandscape = Resources.TestImageLandscapeDesert;

    const int HEIGHT = 300;
    const int WIDTH = 300;
    const int EXPECTED_WIDTH = WIDTH;
    const int EXPECTED_HEIGHT = (int)(EXPECTED_WIDTH / (1024m / 768m));
    const PixelFormat EXPECTED_FORMAT = PixelFormat.Format48bppRgb;
    bool calledBitMapConstructor = false;
    bool calledGraphicsFromImage = false;
    bool calledGraphicsDrawImage = false;

    using (ShimsContext.Create())
    {
        ShimBitmap.ConstructorInt32Int32PixelFormat = (instance, w, h, f) => {
            calledBitMapConstructor = true;
            Assert.AreEqual(EXPECTED_WIDTH, w);
            Assert.AreEqual(EXPECTED_HEIGHT, h);
            Assert.AreEqual(EXPECTED_FORMAT, f);
            ShimsContext.ExecuteWithoutShims(() => {
                ConstructorInfo constructor = typeof(Bitmap).GetConstructor(new[] { typeof(int), typeof(int), typeof(PixelFormat) });
                Assert.IsNotNull(constructor);
                constructor.Invoke(instance, new object[] { w, h, f });
            });
        };
        ShimGraphics.FromImageImage = i => {
            calledGraphicsFromImage = true;
            Assert.IsNotNull(i);
            return ShimsContext.ExecuteWithoutShims(() => Graphics.FromImage(i));
        };
        ShimGraphics.AllInstances.DrawImageImageInt32Int32Int32Int32 = (instance, i, x, y, w, h) => {
            calledGraphicsDrawImage = true;
            Assert.IsNotNull(i);
            Assert.AreEqual(0, x);
            Assert.AreEqual(0, y);
            Assert.AreEqual(EXPECTED_WIDTH, w);
            Assert.AreEqual(EXPECTED_HEIGHT, h);
            ShimsContext.ExecuteWithoutShims(() => instance.DrawImage(i, x, y, w, h));
        };
        using (Image resizedImage = ImageHelper.ResizeImage(testImageLandscape, HEIGHT, WIDTH))
        {
            Assert.IsNotNull(resizedImage);
            Assert.AreEqual(EXPECTED_WIDTH, resizedImage.Size.Width);
            Assert.AreEqual(EXPECTED_HEIGHT, resizedImage.Size.Height);
            Assert.AreEqual(EXPECTED_FORMAT, resizedImage.PixelFormat);
        }
    }

    Assert.IsTrue(calledBitMapConstructor);
    Assert.IsTrue(calledGraphicsFromImage);
    Assert.IsTrue(calledGraphicsDrawImage);
}

score 0 · Answer 2 · answered Jun 11 '18 at 16:29

A bit late to the table with this, but adding this in case it helps anyone out. In my unit tests, this reliably compared images I had dynamically generated using GDI+.

private static bool CompareImages(string source, string expected)
{
    var image1 = new Bitmap($".\\{source}");
    var image2 = new Bitmap($".\\Expected\\{expected}");

    var converter = new ImageConverter();
    var image1Bytes = (byte[])converter.ConvertTo(image1, typeof(byte[]));
    var image2Bytes = (byte[])converter.ConvertTo(image2, typeof(byte[]));

    // ReSharper disable AssignNullToNotNullAttribute
    var same = image1Bytes.SequenceEqual(image2Bytes);
    // ReSharper enable AssignNullToNotNullAttribute

    return same;
}

In .Net is System.Drawing.Image.Save deterministic?

2 Answers2