iOS Cocoa Touch vImage Subsampling

Question

I am working with vImages in Cocoa Touch, which in my case are basically ARGB-float-Arrays, and I need to do a subsampling. Low pass filtering is no problem using the vImage functions but how do I select one out 2x2 pixels (assuming I want to subsample by factor 2)? Of course I could use a vDSP stride function, but this only works for horizontal subsampling, not vertical subsampling.

I hope the following will clarify what I intend to do. I wish to select all pixels marked with an X as shown in this image:

X O X O X O
O O O O O O
X O X O X O
O O O O O O
X O X O X O
O O O O O O

But since the memory is linear, my array looks like this:

X O X O X O O O O O O O X O X O X O O O O O O O X O X O X O O O O O O O

How can I perform subsampling in a reasonable fashion?

EDIT: I am looking for an efficient way to do a two dimensional downsampling of a given bitmap stored as a one dimensional float-array (that includes negative values).

score 1 · Accepted Answer · answered Aug 18 '12 at 13:30

The reality is that when you're doing this sort of subsampling, there isn't really anything clever that can be done; strided memory access doesn't admit many tricks to go fast, so any library code you use will be essentially equivalent to the C code you might write yourself in a few minutes. I would use a simple C implementation.

I believe that this can be done reasonably efficiently on the GPU, but that won't be a performance win unless your data is already in a GPU context. If you have to transfer the data there first, that cost will swamp any performance savings in the actual operation.

David H · Answer 2 · 2012-08-17T10:39:02.637

0

I do this all the time - you need to know the image width. So you have a pointer and you take every other pixel for one row, then you bump the pointer a whole row, then start taking every other pixel til you get to the end of the row, then bump the pointer one whole row again.

For me, I normally have the image rendered into a bit map context, so I know the number of bytes per pixel, the width in pixesl, and the bytes per row (to bump the pointer, but for you it may just be width x bytes-per-pixel).

EDIT: Sorry I was not clear. Given a bitmap of x columns and y rows, you create a new CGBitMapContext, and by using the above sample to take the "every other" pixel and write it to this new context. Now you have a second bitmap if exactly the pixels you say you want in your question. With that bitmap you can now apply whatever further processing you want.

edited Aug 17 '12 at 10:39

answered Aug 16 '12 at 23:31

David H

40,852
12
92
138

I have tried using Core Graphics for this and it works, kind of. Problem is, it does a linear approximation, opposed to the "real" low pass filtering I intend to do. Also I nee to work with negative values, which cannot be displayed as uint8_t. I was hoping to use some of the vDSP functions for better performance instead of implementing a copy loop. As far as I can see this might not be possible for the vertical downsampling but maybe I just don't see it yet. – Chris Aug 17 '12 at 09:15
I edited my response, even thought your comment above does not seem to be included in your original question. You might want to further expand on your question. What I am proposing is not doing any filtering at all, nor using Core Graphics for anything more than hosting the pixels and making an image (which would be an exact rendering of the pixels you create.) – David H Aug 17 '12 at 10:41
I apologize that my question was unclear. Thank you for your answer. I will edit my question with the following: I am looking for an efficient way to do a two dimensional downsampling of a given bitmap stored as a one dimensional float-array (that includes negative values). – Chris Aug 17 '12 at 11:34
I'm more confused than ever! The way the data is stored does not seem to be relevant - if you know how its formatted its more or less the same as if it was a CGBitMapContext. How is it that the algorithm above does not meet this? And why is the negative value relevant? If you have them now, you would have them in your final image, and I see to recall that Quartz clips negative values to 0 and values > 1.0 to 1.0. [Obviously others are confused too, that's why I'm the only person responding!] – David H Aug 17 '12 at 11:49
The format of the data is relevant because I believe this limits the functions I could use. Again I am hoping to use vDSP functions with stride to accomplish this. I don't want to select every pixel "by hand" because that would be rather time consuming. The algorithm must not clip negative values. My output should be another float-array with a quarter of the original length that only contains to elements marked with an X. I am basically trying to build a laplacian image pyramid if that clarifies what I'm trying to do. – Chris Aug 17 '12 at 12:53
From my perspective I've answered your question perfectly. "By Hand" I mean by code you write, not using Accelerate. In any case, if you don't want to clip your image (and is is grayscale or color), you will have to determine the range between largest and smallest value (negative), then apply both a shift and scale so as to not lose anything (at least if you want to display the image using Quartz. In any case, it appears I've said all I can say - good luck! – David H Aug 17 '12 at 13:00

score 0 · Answer 3 · answered Nov 30 '12 at 20:09

Since the memory is linear, lets interpret the source matrix as matrix of double width and half height:

X O X O X O  O O O O O O
X O X O X O  O O O O O O
X O X O X O  O O O O O O

After copying with interleaving (stride = 2) we have:

X X X  O O O
X X X  O O O
X X X  O O O

Then use vDSP_mmov to copy left side submatrix to the result submatrix.

For the copying with interleaving (first phase) can be used vDSP_zvmov function, but it is not optimized for floats. Probably vDSP_vsadd with 0 as scalar param will work faster.

score 0 · Answer 4 · answered Jan 21 '14 at 02:16

I realize that this is mostly just a "how to use pointers" question. However, the operation you are planning to do is not going to result in a high quality image. It will look pixelated and some details will be magnified and others lost.

Fortunately, image resampling techniques are a well researched area, and there are a lot of options here.

If you are on the GPU, the operation you describe is simply the nearest sampling method in OpenCL or OpenGL, so if you are there, you can just calculate the coordinate you want and pick nearest. The linear sampling method will probably look better.

If you are on the CPU, then you have some options. If the data is held in a CGImageRef, you can render it into a CGBitmapContextRef with scaling to reduce the size of the image by a factor of two in each dimension. I believe this gives you linear resampling, but that detail should be available somewhere.

If the data is sitting in a C array, then vImageScale_ARGBFFFF() is your friend. Just create a new buffer 1/2 the height and 1/2 the width of the original and use that function to downsample it to the right values. This will use Lanczos resampling which is a bit slower but looks better. (It is probably more appropriate for floating-point data, which I assumed you picked because you wanted high fidelity.) The Lanczos kernel has the advantage of combining low pass filtering and downsampling into a single pass, so there is less to do.

iOS Cocoa Touch vImage Subsampling

4 Answers4