WinML onnx example produce wrong results

Question

I am following this WinML example to create a regression model and perform inference on it. I am tried running an onnx model using WinML/WinRT, but the result comes out wrong. I enforced that the data is read as RGB not BGR, but there is an alpha component included, e.g. RGBA, I suspect that the wrong result is due to the alpha, which I do not have it in my original model. How do I come around this problem? Code snippet Console output

VideoFrame LoadImageFile(hstring filePath, ColorManagementMode colorManagementMode)
{
    BitmapDecoder decoder = NULL;
    try
    {
        // open the file
        StorageFile file = StorageFile::GetFileFromPathAsync(filePath).get();
        // get a stream on it
        auto stream = file.OpenAsync(FileAccessMode::Read).get();
        // Create the decoder from the stream
        decoder = BitmapDecoder::CreateAsync(stream).get();
        decoder.GetPixelDataAsync().get().DetachPixelData()[3] = 1;
        decoder.GetPixelDataAsync().get().DetachPixelData()[7] = 1;
        auto pix = decoder.GetPixelDataAsync(BitmapPixelFormat::Rgba8, BitmapAlphaMode::Ignore, BitmapTransform(), ExifOrientationMode::IgnoreExifOrientation, ColorManagementMode::DoNotColorManage);
        for (auto b : pix.get().DetachPixelData())printf("%d\n", b);
       // printf("my pixels: %d", pix.get().DetachPixelData();
    }
    catch (...)
    {
        printf("    Failed to load the image file, make sure you are using fully qualified paths\r\n");
        exit(EXIT_FAILURE);
    }
    SoftwareBitmap softwareBitmap = NULL;
    try
    {
        softwareBitmap = decoder.GetSoftwareBitmapAsync(
            //decoder.BitmapPixelFormat(),
            BitmapPixelFormat::Rgba8,
            //decoder.BitmapAlphaMode(),
            BitmapAlphaMode::Ignore,
            BitmapTransform(),
            ExifOrientationMode::RespectExifOrientation,
            colorManagementMode
        ).get();
        
        printf("Image format: %d\n", softwareBitmap.BitmapPixelFormat());
    }

when I read the pixels from the decoder, I get RGBA, where A (alpha), is set to 255. I tried to replace it with 1, but it appears the that decoder is immutable. If I can ensure that the pixels fed to model are correct, then this will produce the correct result.

Answer 1

Please Just follow the example to create VideoFrame, you don't need to worry about the alpha channel when creating videoFrame because WinML will ignore this channel anyway. And if needed, call SoftwareBitmap.Convert() to convert between different pixel formats

VideoFrame LoadImageFile(hstring filePath)
{
    printf("Loading the image...\n");
    DWORD ticks = GetTickCount();
    VideoFrame inputImage = nullptr;

    try
    {
        // open the file
        StorageFile file = StorageFile::GetFileFromPathAsync(filePath).get();
        // get a stream on it
        auto stream = file.OpenAsync(FileAccessMode::Read).get();
        // Create the decoder from the stream
        BitmapDecoder decoder = BitmapDecoder::CreateAsync(stream).get();
        // get the bitmap
        SoftwareBitmap softwareBitmap = decoder.GetSoftwareBitmapAsync().get();
        // load a videoframe from it
        inputImage = VideoFrame::CreateWithSoftwareBitmap(softwareBitmap);
    }
    catch (...)
    {
        printf("failed to load the image file, make sure you are using fully qualified paths\r\n");
        exit(EXIT_FAILURE);
    }

    ticks = GetTickCount() - ticks;
    printf("image file loaded in %d ticks\n", ticks);
    // all done
    return inputImage;
}