How can I flip the Kinect v2 colorstream horizontally?

Question

I recently set-up my Kinect v2 on my win 10 PC. Intending to use it for some VR mixed reality streaming I have found that the video feed is mirrored and there is no way to flip it.

I have attempted to modify the Coordinate Mapping Basics program in the SDK to be a green screen and to flip the image. The green I have done but I can not get the colorstream to flip!

I have tried turning the WriteableBitmap to a bitmap, exporting it an array then flipping the array and writing it back to a WritableBitmap but that didn't work.

I have tried installing WriteableBitmapEx library and using the flip function but the program just freezes.

Any suggestions?

Long term I also want the program to act as a virtual webcam to pass the video feed straight to a compositor, but one step at a time.

UPDATE: I Have got the image to flip BUT its at a HUGE performance hit. Wrote this to convert the WriteableBitmap into a byte Array then reverse the order. Doing some tests its the loop which is killing the performance. Any alternatives or ways to optimise the loop?

private void flipBitmap(int method)
    {            
        if (method == 1)//works but slow.
        {
            var width = this.bitmap.PixelWidth;
            var height = this.bitmap.PixelHeight;
            var stride = ((PixelFormats.Bgr32.BitsPerPixel + 7) / 8) * width;
            int current, flipped = 0;
            var raw= new byte[stride * height];
            this.bitmap.CopyPixels(bitmapData, stride, 0);
            var flippedPixels = new byte[raw.Length];
            for (int y = 0; y < height; y++)
            {

                for (int x = 4; x < width; x++)
                {
                    current = (y * stride) + (x * 4);
                    flipped = (y * stride) + (width - x) * 4;
                    for (int i = 0; i < 3; i++)
                    {
                        flippedPixels[flipped + i] = raw[current + i];
                    }
                    flippedPixels[flipped + 3] = 255;
                }
            }
            this.bitmap.WritePixels(new Int32Rect(0, 0, width, height), flippedPixels, stride, 0);
        }
        else if(method == 2)//placeholder
        {
            return;
        }
        else 
        {
            return;
        }                        
    }

UPDATE 2: I now have it running at what looks like 30fps. Just got to get the performance better! I pulled the variables out so it was not making new arrays all the time, just recycling the old ones. Now thinking about running parallel for loops, one going from the start of the array, the other from the end. No idea if that will speed it up or use more/less processing power. Right now the program uses 40-50% of my cpu which is insane and much more than I can do as well as VR. Still interested in faster/more efficient ways to flip the image if anyone has them.

UPDATE 3: With A little more tinkering and cleaning of old experiments the performance has improved but its still using twice the cpu the nonflipped version uses.

private void flipBitmap(int method)
    {

        if (method == 1)//works but cpu intensive.
        {           
            int current, flipped = 0;
            this.bitmap.CopyPixels(raw, stride, 0);                

            for (int y = 0; y < height; y++)
            {

                for (int x = 4; x < width; x++)
                {
                    current = (y * stride) + (x * 4);
                    flipped = (y * stride) + (width - x) * 4;

                    flippedPixels[flipped + 0] = raw[current + 0];
                    flippedPixels[flipped + 1] = raw[current + 1];
                    flippedPixels[flipped + 2] = raw[current + 2];
                    flippedPixels[flipped + 3] = 255;                        
                }
            }
            this.bitmap.WritePixels(new Int32Rect(0, 0, width, height), flippedPixels, stride, 0);

            return;
        }
        else if(method == 2)//placeholder
        {               
            return;
        }
        else 
        {                
            return;
        }                        
    }

It's probably best to just leave the image data as it is and reflect it at the point of use. For example, if rendering then you can flip the UV coordinates on the shader. If accessing indivdual pixels then you can create a wrapper function that returns the pixel at `width - x` when requesting pixel `x`. Other than that I would guess that using a library like OpenCV will be much quicker than a for loop as it probably has a vectorized implementation — Tim MB, May 01 '20 at 21:00

score 0 · Answer 1 · answered Jun 10 '18 at 16:06

0

Have you tried this? https://msdn.microsoft.com/en-us/library/system.drawing.image.rotateflip(v=vs.110).aspx

It looks like exactly what you're looking for...

answered Jun 10 '18 at 16:06

David Culbreth

2,610
16
26

After thinking about it, this isn't a trivial operation, so the most obvious way to get a higher frame rate would be to use a GPU and library in postprocessing. I would write an answer to integrate this solution, but that's a task that consumes hundreds of people's lives for virtually every piece of graphical software. – David Culbreth Jun 10 '18 at 16:46
writeableBitmap doesn't have rotateflip on it. Might try converting to bitmap then flipping then changing back. I was thinking if there was a way to push the load to the GPU but I have never touched something like that before so was trying to stick to things I had at least some idea about. – trekimann Jun 10 '18 at 16:47

How can I flip the Kinect v2 colorstream horizontally?

1 Answers1