1

In any image processing library there is always the issue of having to provide an implementation of every single algorithm for every single image format (color space, channels, bit depth, mem layout, etc). One very elegant solution to the problem is Boost GIL. Through the power of C++, and a great design, all of these issues are abstracted away and you can write a single algorithm that will work on any type of image.

I wanted to create something similar in C#, but many of the necessary constructs such as templates and certain operator overloads (like unary *) are missing. I'm willing to accept that what I can create will not be nearly as robust and elegant as GIL, but to the extent possible I would like to model the concepts. Ultimately, abstracting image differences and writing generic processing algorithms would be the aim.

With what there is available in C#, generics, lambdas, even dynamic IL /cringe, what do people think some possible approaches would be to designing the library?

kennytm
  • 510,854
  • 105
  • 1,084
  • 1,005
Charles
  • 53
  • 4
  • I'm not sure what you mean exactly, but we have to deal with a lot of different image formats in our project and we simply do so by using the managed extensibility framework. – TimothyP Nov 26 '09 at 10:07
  • MEF is cool, but I'm not sure it addresses what I am saying. Can you take a look at GIL? http://www.boost.org/doc/libs/1_39_0/libs/gil/doc/index.html. It literally lets you write one algorithm to write on any image format. Image an iterator that could enumerate every pixel regardless of number of channels or bit depth, or even whether the channels were laid out in interleaved or planar order. This and a lot more is done in GIL, but at near hand-coded native speeds because of templates. – Charles Nov 26 '09 at 18:46
  • I guess to be more clear: say I have an algorithm to calculate a histogram. I do not want to have to write a separate one for grayscale, 565 RGB, 24 RGB, 32 RGB, RGBA, ARGB, BGRA, ABGR, or even other color spaces like CYMK and HSL. Through the magic of software these differences should be able to be abstracted so the algorithm writer of an image processing routine only has to write -one- algorithm that will operate across all formats. – Charles Nov 26 '09 at 18:51

2 Answers2

0

Have you seen Aforge.NET, the way that's designed is pretty generic.

The author of that library solved a lot of the problems you are talking about through interfaces. Off the top of my head stuff like, IFilter, IFilterColourToAny etc

To do efficient image processing in .NET you'll need to go into unsafe code using Bitmap.LockData (I think), which could negate all the cool .NET stuff you're talking about...

ParmesanCodice
  • 5,017
  • 1
  • 23
  • 20
  • This is a great library, but they don't generalize any of the image processing routines to be agnostic of format. Everything they have works with an RGB bitmap or a grayscale image. I'm looking for a way to generalize iterating over pixels and pixel formats like GIL does. In GIL you can write one algorithm for something like thresholding, for example, and it can work on any image in any color space, with any number of channels, with any number of bits per channel. That's ultimately what I am trying to accomplish. – Charles Nov 26 '09 at 18:17
  • @Charles: I wish this were doable in C#. I have yet to find a way. C++ templates are just *far* more powerful than C# generics. C# really falls on its face when it comes to generic number crunching. – Ed S. Aug 07 '13 at 21:33
0

Though my answer comes awfully late, I hope it will be useful for other people.

Given the limitations of C# that the OP has identified so far, here is a list of criteria that would still give limited freedom to the programmer in terms of pixel formats.

  • A flexible bitmap handle C# class, which supports a variety of pixel formats without being a C# generic class
    • Because the generic class in the C# language would not have afforded any benefit to the user of image processing.
  • Some safety guarantee - certain characteristics of a bitmap handle object should be immutable
    • In particular, the bitmap's pixel format and pixel dimensions should not be allowed to change once created. To change these, a new object must be created.
  • Provides both mutable interface (allows change to pixel values) and immutable object model interface
    • To provide an indication on any API function that there should be no attempt to write to a particular bitmap argument.
  • A set of image processing algorithm classes that accept and return the bitmap handle class.
    • Each algorithm tries its best to handle different kinds of pixel formats, to the extent that is economically feasible.

With the criteria in mind, I would recommend using System.Windows.Media.Imaging as the substrate of the library you are building.

The System.Windows.Media.Imaging namespace is the C# counterpart to the Microsoft Windows Imaging Component (WIC) library. Therefore, the underlying processing is implemented in native C++, giving it the speed needed for practical use.

Thanks to the wide range of pixel format support implemented in WIC, the C# counterpart also supports the same range of pixel formats.

The WIC (and System.Windows.Media.Imaging) does not provide any advanced image processing capability (nothing of the sort of Canny edge detection, Hough transform, object detection, etc.)

However, in terms of being an in-memory bitmap object interchange interface (for integrating different image libraries with C# interfaces or bindings), both System.Windows.Media.Imaging.WriteableBitmap and System.Drawing.Bitmap are suitable.

In terms of implementing algorithms, it is sometimes difficult to make algorithms equally applicable to single-channel images and multi-channel images. Such requires years if not decades of research in multivariate mathematics.

Thus, it is common for image processing algorithm classes to focus on supporting a narrow subset of pixel formats:

  • 1bpp Black White (for logical / decision bitmaps, e.g. edge map or connected component blob membership)
  • 8bpp Gray
  • 24bpp BGR
  • 32bpp BGRA
  • 32bpp Gray Float
  • 96bpp BGR Float
  • 128bpp BGRA Float

If an algorithm class sees an input bitmap handle that is not one of the above type, it would try its best to losslessly "promote" the input format into one of the above formats.

With this automatic up-conversion in mind, the user of this algorithm class loses strict control on the output bitmap's pixel format, but gains the guarantee that the output's visual appearance will be in accordance with expectation.

rwong
  • 6,062
  • 1
  • 23
  • 51