0

I have a device creating raw frames and placing them in memory pointed to as a uint32_t* that is allocated for the frame size. The data is packed bytes with no padding as 1460 pixels, 1920 lines and 2 bytes per pixel. I am trying to display the frame(s) using openCV. I have created the frame with the following line: cv::Mat frame(1460, 1920, CV_16UC1, g_mem1);

When this called with: cv::imshow("Video Playback", frame); I get a distorted image. If I perform the following operations on the pixels:

size_t memSz = 1920 * 1460 * 2;
uint32_t* g_mem1 = static_cast<uint32_t>(std::malloc(memSz));
cv::Mat frame(1460, 1920, CV_16UC1, g_mem1);

for (int y = 0; y < 1460; ++y)
{
    for (int x = 0; x < (1920 / 2); ++x)
    {
        uint32_t data = g_mem1[(y * (1920 / 2) + x)];
        uint16_t pixelValue1 = static_cast<uint16_t>(data>> 16);
        pixelValue1 = (pixelValue1 >> 8) | (pixelValue1 << 8);

        uint16_t pixelValue2 = static_cast<uint16_t>(data & 0xFFFF);
        pixelValue2 = (pixelValue2 >> 8) | (pixelValue2 << 8);
        frame.at<uint16_t>(y, (x * 2)) = pixelValue1;
        frame.at<uint16_t>(y, (x * 2) + 1) = pixelValue2;
    }
}

The images now displays correctly. This is much too slow to be doing in software. I have a working pipeline with gstreamer that uses a cap format of gray16-be that displays the data correctly with no manipulation. I can't use gstreamer on the target machine and I am trying to find the openCV equivalent that does not involve manipulating the pixel data or at least lets me leverage hardware to do so.

I have tried the various CV_16 formats available. The data still appeared distorted and only manually manipulating the data as described above has resulted in valid output using openCV.

Pasonis
  • 1
  • 1
  • 1
    Provide a proper MCVE -- you claim that `frame` is a `Mat` of `CV_16UC1`, yet the first thing you do in the loop is `frame[(y * (1920 / 2) + x)]`. How does that work, given that `cv::Mat` doesn't even define `operator[]`? Even if it did work, how would that return a 32bit integer? Yet few lines later you do `frame.at(y, (x * 2))`, which is clearly a call to a method of `cv::Mat`. – Dan Mašek Aug 29 '23 at 11:36
  • 3
    Anyway, the problem is really converting 16bit values from big endian to little endian -- i.e. byte swapping. Create a temporary `Mat` header to treat the data as `CV_8UC2`. Use `cv::mixChannels` to swap the byte (in this case also channel) order in-place. – Dan Mašek Aug 29 '23 at 11:48
  • Sorry for the typo, instead of frame that was supposed to be g_mem1 which is the memory buffer that was allocated and is of type uint32_t*. I was trying to pare down the code snippets to be as small as possible. Can you explain the call to cv::mixChannels. I am new to openCV. – Pasonis Aug 29 '23 at 12:58
  • I attempted the following but it crashed, stating that there was an invalid number of channels in input image: `curFrame = cv::Mat(height, width, CV_8UC2, g_mem1);` `int fromTo[] = { 1, 0, 0, 1 };` `cv::mixChannels(&curFrame, 1, &curFrame, 1, fromTo, 2);` – Pasonis Aug 29 '23 at 13:06
  • Ah, unfortunately it seems that `mixChannels` can't work in-place. Here are two variations, depending on how much swapping you really need: https://pastebin.com/XeWNTRTg – Dan Mašek Aug 29 '23 at 13:57

0 Answers0