C# search for a bitmap within another bitmap

Question

I've tried open source projects such as this one however it doesn't seem to work at all for me. I then attempted to write my own algorithm like so (tolerance isn't being used yet).

public static Rectangle ImageSearch(Bitmap ToSearch, Bitmap ToFind, int Tolerance, double MinPercent) {
        Rectangle ReturnValue = Rectangle.Empty;
        BitmapData ToSearchData = ToSearch.LockBits(new Rectangle(0, 0, ToSearch.Width, ToSearch.Height), ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
        BitmapData ToFindData = ToFind.LockBits(new Rectangle(0, 0, ToFind.Width, ToFind.Height), ImageLockMode.ReadOnly, PixelFormat.Format24bppRgb);
        IntPtr ToSearchScan0 = ToSearchData.Scan0;
        IntPtr ToFindScan0 = ToFindData.Scan0;
        int PixelWidth = 3; // 3 since 24 bits per pixel format
        int ToSearchStride = ToSearchData.Stride;
        int ToSearchPadding = ToSearchStride - (ToSearch.Width * PixelWidth);
        int ToFindStride = ToFindData.Stride;
        int ToFindPadding = ToFindStride - (ToFind.Width * PixelWidth);

        unsafe {
            byte *ToSearchPixelArray = (byte*)(void*)ToSearchData.Scan0;
            byte *ToFindPixelArray = (byte*)(void*)ToFindData.Scan0;
            byte sB, sG, sR, fB, fG, fR;

            fB = ToFindPixelArray[0];
            fG = ToFindPixelArray[1];
            fR = ToFindPixelArray[2];

            for (int sY = 0; sY < ToSearch.Height; sY++) {
                for (int sX = 0; sX < ToSearch.Width * PixelWidth; sX += PixelWidth) {
                    sB = ToSearchPixelArray[0];
                    sG = ToSearchPixelArray[1];
                    sR = ToSearchPixelArray[2];

                    if (sB == fB && sG == fG && sR == fR) {
                        Console.WriteLine("found possible match");

                        byte *ToSearchBackup = ToSearchPixelArray;
                        byte *ToFindBackup = ToFindPixelArray;
                        int MatchedPixels = 0;

                        for (int fY = 0; fY < ToFind.Height; fY++) {
                            for (int fX = 0; fX < ToFind.Width * PixelWidth; fX += PixelWidth) {
                                fB = ToFindPixelArray[0];
                                fG = ToFindPixelArray[1];
                                fR = ToFindPixelArray[2];
                                sB = ToSearchPixelArray[0];
                                sG = ToSearchPixelArray[1];
                                sR = ToSearchPixelArray[2];

                                if (sB == fB && sG == fG && sR == fR) {
                                    ++MatchedPixels;
                                } else {
                                    ToSearchPixelArray = ToSearchBackup;
                                    ToFindPixelArray = ToFindBackup;

                                    // this is the best way to break a nested loop in C#
                                    fX = int.MaxValue;
                                    fY = int.MaxValue; 
                                }
                            }

                            ToSearchPixelArray += ToSearchStride - sX;
                            ToFindPixelArray += ToFindPadding;
                        }

                        if (MatchedPixels / (ToFind.Width * ToFind.Height) >= MinPercent) {
                            ReturnValue.X = (int)(sX / 3);
                            ReturnValue.Y = sY;
                            ReturnValue.Width = ToFind.Width;
                            ReturnValue.Height = ToFind.Height;

                            // this is the best way to break a nested loop in C#
                            sX = int.MaxValue;
                            sY = int.MaxValue; 
                        }
                    }
                }

                ToSearchPixelArray += ToSearchPadding;
            }
        }

        ToSearch.UnlockBits(ToSearchData);
        ToFind.UnlockBits(ToFindData);

        return ReturnValue;
    }

But not even this will detect a screenshot I take of the exact image I'm searching through. Please do not suggest things such as Emgu, I'm using this in a commercial application and cannot afford to purchase a license from any GNU licensed projects (I'm not open sourcing the project either).

Have you tried using Paint to create something like a 2x2 bitmap and searching for a single pixel target? You should easily be able to step through that to see where it's going wrong. — Bradley Uffner, Feb 18 '15 at 15:10
@BradleyUffner That's a very good idea, thank you. It's been hard to debug on the image I'm using since it's so large but creating a small bitmap for testing is probably the best way to go. — AnthonyOSX, Feb 18 '15 at 15:14

BrilllianD · Answer 1 · 2015-03-31T21:17:53.943

Serching many entries "serchingBitmap" in "sourceBitmap". In this one I don't using unsafe code.

public static List<Point> FindBitmapsEntry(Bitmap sourceBitmap, Bitmap serchingBitmap)
    {
        #region Arguments check

        if (sourceBitmap == null || serchingBitmap == null)
            throw new ArgumentNullException();

        if (sourceBitmap.PixelFormat != serchingBitmap.PixelFormat)
            throw new ArgumentException("Pixel formats arn't equal");

        if (sourceBitmap.Width < serchingBitmap.Width || sourceBitmap.Height < serchingBitmap.Height)
            throw new ArgumentException("Size of serchingBitmap bigger then sourceBitmap");

        #endregion

        var pixelFormatSize = Image.GetPixelFormatSize(sourceBitmap.PixelFormat)/8;


        // Copy sourceBitmap to byte array
        var sourceBitmapData = sourceBitmap.LockBits(new Rectangle(0, 0, sourceBitmap.Width, sourceBitmap.Height),
            ImageLockMode.ReadOnly, sourceBitmap.PixelFormat);
        var sourceBitmapBytesLength = sourceBitmapData.Stride * sourceBitmap.Height;
        var sourceBytes = new byte[sourceBitmapBytesLength];
        Marshal.Copy(sourceBitmapData.Scan0, sourceBytes, 0, sourceBitmapBytesLength);
        sourceBitmap.UnlockBits(sourceBitmapData);

        // Copy serchingBitmap to byte array
        var serchingBitmapData =
            serchingBitmap.LockBits(new Rectangle(0, 0, serchingBitmap.Width, serchingBitmap.Height),
                ImageLockMode.ReadOnly, serchingBitmap.PixelFormat);
        var serchingBitmapBytesLength = serchingBitmapData.Stride * serchingBitmap.Height;
        var serchingBytes = new byte[serchingBitmapBytesLength];
        Marshal.Copy(serchingBitmapData.Scan0, serchingBytes, 0, serchingBitmapBytesLength);
        serchingBitmap.UnlockBits(serchingBitmapData);

        var pointsList = new List<Point>();

        // Serching entries
        // minimazing serching zone
        // sourceBitmap.Height - serchingBitmap.Height + 1
        for (var mainY = 0; mainY < sourceBitmap.Height - serchingBitmap.Height + 1; mainY++)
        {
            var sourceY = mainY * sourceBitmapData.Stride;

            for (var mainX = 0; mainX < sourceBitmap.Width - serchingBitmap.Width + 1; mainX++)
            {// mainY & mainX - pixel coordinates of sourceBitmap
                // sourceY + sourceX = pointer in array sourceBitmap bytes
                var sourceX = mainX*pixelFormatSize;

                var isEqual = true;
                for (var c = 0; c < pixelFormatSize; c++)
                {// through the bytes in pixel
                    if (sourceBytes[sourceX + sourceY + c] == serchingBytes[c]) 
                        continue;
                    isEqual = false;
                    break;
                }

                if (!isEqual) continue;

                var isStop = false;

                // find fist equalation and now we go deeper) 
                for (var secY = 0; secY < serchingBitmap.Height; secY++)
                {
                    var serchY = secY * serchingBitmapData.Stride;

                    var sourceSecY = (mainY + secY)*sourceBitmapData.Stride;

                    for (var secX = 0; secX < serchingBitmap.Width; secX++)
                    {// secX & secY - coordinates of serchingBitmap
                        // serchX + serchY = pointer in array serchingBitmap bytes

                        var serchX = secX*pixelFormatSize;

                        var sourceSecX = (mainX + secX)*pixelFormatSize;

                        for (var c = 0; c < pixelFormatSize; c++)
                        {// through the bytes in pixel
                            if (sourceBytes[sourceSecX + sourceSecY + c] == serchingBytes[serchX + serchY + c]) continue;

                            // not equal - abort iteration
                            isStop = true;
                            break;
                        }

                        if (isStop) break;
                    }

                    if (isStop) break;
                }

                if (!isStop)
                {// serching bitmap is founded!!
                    pointsList.Add(new Point(mainX, mainY));
                }
            }
        }

        return pointsList;
    }

Good, but some people here don't understand Russian. Translate your comments, please. — Dmitriy, Mar 29 '15 at 20:15

score 0 · Answer 2 · answered Apr 15 '20 at 13:28

It doesn't work accurately for me but it does give me an idea. I think the problem with this soludion is that it is looking for an exact pixel-for-pixel instance. Basically I am doing what you are doing, trying to find 1 or more occurrences of a bitmap in another but the properties may vary like brightness, contrast, size, etc. I have tired several things including Aforge.Net and Accord.Net but I can't seem to get an acceptable accuracy > 50%. Thanks for posting.

C# search for a bitmap within another bitmap

2 Answers2