DirectShow universal media decoder

Question

I am new to DirectShow API.

I want to decode a media file and get uncompressed RGB video frames using DirectShow.

I noted that all such operations should be completed through a GraphBuilder. Also, every the processing block is called a filter and there are many different filters for different media files. For example, for decoding H264 we should use "Microsoft MPEG-2 Video Decoder", for AVI files "AVI Splitter Filter" etc.

I would like to know if there is a general way (decoder) that can handle all those file types?

I would really appreciate if someone can point out an example that goes from importing a local file to decoding it into uncompressed RGB frames. All the examples I found are dealing with window handles and they just configure it and call pGraph->run(). I have also surfed through Windows SDK samples, but couldn't find useful samples.

Thanks very much in advance.

Answer 1

Universal DirectShow decoder in general is against the concept of DirectShow API. The whole idea is that individual filters are responsible for individual task (esp. decoding certain encoding or demultiplexing certain container format). The registry of the filters and Intelligent Connect let one to have the filters built in chain to do certain requested processing, in particular decoding from compressed format to 24-bit RGB for video.

From this standpoint you don't need a universal decoder and it is not expected that such decoder exists. However, such decoder (or close) does exist and it's a ffdshow or one of its derivatives. Presently, you might want to look at LAVFilters, for example. They wrap FFmpeg, which itself can handle many formats, and connect it to DirectShow API so that, as as filter, ffdshow could handle many formats/encodings.

There is no general rule to use or not use such codec pack, in most cases you take into consideration various factors and decide what to do. If your application handles various scenarios, a good starting point into graph building would be Overview of Graph Building.

My goal is to accomplish the task using DirectShow in order to have no external dependencies. Do you know a particular example that does uncompressing frames for some file type?

Your request is too broad and in the same time typical and, to some extent, fairy simple. If you spend some time playing with GraphEdit SDK tool, or rather GraphStudioNext, which is a more powerful version of the former, you will be able to build filter graph interactively, also render media files of different types and see what filters participate in rendering. You can accomplish the very same programmatically too, since the interactive actions basically all have matching API calls individually.

You will be able to see that specific formats are handled by different filters and Intelligent Connect mentioned above is building chains of filters in combinations in order to satisfy the requests and get the pipeline together.

Default use case is playback, and if you want to get video rendered to 24/32-bit RGB, your course of actions is pretty much similar: you are to build a graph, which just needs to terminate with something else. More flexible, sophisticated and typical for advanced development approach is to supply a custom video renderer filter and accept decompressed RGB frames on it.

A simple and so much popular version of the solution is to use Sample Grabber filter, initialize it to accept RGB, setup a callback on it so that your SampleCB callback method is called every time RGB frame is decompressed, and use Sample Grabber in the graph. (You will find really a lot of attempts to accomplish that if you search open source code and/or web for keywords ISampleGrabber, ISampleGrabberCB, SampleCB or BufferCB, MEDIASUBTYPE_RGB24).

• Using the Sample Grabber
• DirectShow: Examples for Using SampleGrabber for Grabbing a Frame and Building a VU Meter

Another more or less popular approach is to setup a playback pipeline, play a file, and read back frames from video presenter. This is suggested in another answer to the question, is relatively easy to do, and does the job if you don't have performance requirement and requirements to extract every single frame. That is, it is a good way to get a random RGB frame from the feed but not every/all frames. See related on this:

• Different approaches on getting captured video frames in DirectShow

Answer 2

You are looking for vmr9 example in DirectShow library.

In your Windows SDK's install, look for this example:

Microsoft SDKs\Windows\v7.0\Samples\multimedia\directshow\vmr9\windowless\windowless.sln

And search this function: CaptureImage, in this method, see IVMRWindowlessControl9::GetCurrentImage, is exactly what you want.

This method captures a video frame in bitmap format (RGB). Next, this is a copy of CaptureImage code:

BOOL CaptureImage(LPCTSTR szFile)
{
HRESULT hr;

if(pWC && !g_bAudioOnly)
{
    BYTE* lpCurrImage = NULL;

    // Read the current video frame into a byte buffer.  The information
    // will be returned in a packed Windows DIB and will be allocated
    // by the VMR.
    if(SUCCEEDED(hr = pWC->GetCurrentImage(&lpCurrImage)))
    {
        BITMAPFILEHEADER    hdr;
        DWORD               dwSize, dwWritten;
        LPBITMAPINFOHEADER  pdib = (LPBITMAPINFOHEADER) lpCurrImage;

        // Create a new file to store the bitmap data
        HANDLE hFile = CreateFile(szFile, GENERIC_WRITE, FILE_SHARE_READ, NULL,
                                  CREATE_ALWAYS, FILE_ATTRIBUTE_NORMAL, 0);

        if (hFile == INVALID_HANDLE_VALUE)
            return FALSE;

        // Initialize the bitmap header
        dwSize = DibSize(pdib);
        hdr.bfType          = BFT_BITMAP;
        hdr.bfSize          = dwSize + sizeof(BITMAPFILEHEADER);
        hdr.bfReserved1     = 0;
        hdr.bfReserved2     = 0;
        hdr.bfOffBits       = (DWORD)sizeof(BITMAPFILEHEADER) + pdib->biSize +
            DibPaletteSize(pdib);

        // Write the bitmap header and bitmap bits to the file
        WriteFile(hFile, (LPCVOID) &hdr, sizeof(BITMAPFILEHEADER), &dwWritten, 0);
        WriteFile(hFile, (LPCVOID) pdib, dwSize, &dwWritten, 0);

        // Close the file
        CloseHandle(hFile);

        // The app must free the image data returned from GetCurrentImage()
        CoTaskMemFree(lpCurrImage);

        // Give user feedback that the write has completed
        TCHAR szDir[MAX_PATH];

        GetCurrentDirectory(MAX_PATH, szDir);

        // Strip off the trailing slash, if it exists
        int nLength = (int) _tcslen(szDir);
        if (szDir[nLength-1] == TEXT('\\'))
            szDir[nLength-1] = TEXT('\0');

        Msg(TEXT("Captured current image to %s\\%s."), szDir, szFile);
        return TRUE;
    }
    else
    {
        Msg(TEXT("Failed to capture image!  hr=0x%x"), hr);
        return FALSE;
    }
}

return FALSE;
}