4

I try to encode images of my desktop (1920x1080) to a video file using DXGI Desktop Duplication API and MF Sink Writer. My encoding thread looks like this:

#define RETURN_ON_BAD_HR(expr) \
{ \
    HRESULT _hr_ = (expr); \
    if (FAILED(_hr_)) { \
        qDebug() << "Error encountered with message from HRESULT: " << \
                    getMessageFromHR(_hr_); \
        MFShutdown(); \
        CoUninitialize(); \
        return; \
    } \
}

void DuplicationThread::run() {
    if (CoInitializeEx(NULL, COINIT_APARTMENTTHREADED) != S_OK) {
        qDebug() << "Failed to initialize COM.";
        return;
    }

    if (MFStartup(MF_VERSION) != S_OK) {
        qDebug() << "Failed to initialize Media Foundation.";
        CoUninitialize();
        return;
    }

    CComPtr<ID3D11Device> pDevice;
    RETURN_ON_BAD_HR(InitializeDx(&pDevice));

    CComPtr<IDXGIOutputDuplication> pDeskDupl;
    RETURN_ON_BAD_HR(InitiateDupl(pDevice, &pDeskDupl));

    CComPtr<IMFSinkWriter> pSinkWriter;
    DWORD streamIndex;
    RETURN_ON_BAD_HR(InitSinkWriter(&pSinkWriter, &streamIndex));

    LONGLONG rtStart = 0;

    while (!isInterruptionRequested()) {
        DXGI_OUTDUPL_FRAME_INFO frameInfo;
        CComPtr<IDXGIResource> pDesktopResource;
        RETURN_ON_BAD_HR(pDeskDupl->AcquireNextFrame(500, &frameInfo, &pDesktopResource));

        CComPtr<ID3D11Texture2D> pAcquiredDesktopImage;
        RETURN_ON_BAD_HR(pDesktopResource->QueryInterface(__uuidof(ID3D11Texture2D), reinterpret_cast<void **>(&pAcquiredDesktopImage)));

        CComPtr<IMFMediaBuffer> pMediaBuffer;
        RETURN_ON_BAD_HR(MFCreateDXGISurfaceBuffer(__uuidof(ID3D11Texture2D), pAcquiredDesktopImage, 0, FALSE, &pMediaBuffer));

        CComPtr<IMF2DBuffer> p2DBuffer;
        DWORD length;
        RETURN_ON_BAD_HR(pMediaBuffer->QueryInterface(__uuidof(IMF2DBuffer), reinterpret_cast<void **>(&p2DBuffer)));
        RETURN_ON_BAD_HR(p2DBuffer->GetContiguousLength(&length));
        RETURN_ON_BAD_HR(pMediaBuffer->SetCurrentLength(length));

        CComPtr<IMFSample> pSample;
        RETURN_ON_BAD_HR(MFCreateVideoSampleFromSurface(NULL, &pSample));
        RETURN_ON_BAD_HR(pSample->AddBuffer(pMediaBuffer));
        RETURN_ON_BAD_HR(pSample->SetSampleTime(rtStart));
        RETURN_ON_BAD_HR(pSample->SetSampleDuration(VIDEO_FRAME_DURATION));

        RETURN_ON_BAD_HR(pSinkWriter->WriteSample(streamIndex, pSample));
        RETURN_ON_BAD_HR(pDeskDupl->ReleaseFrame());
        rtStart += VIDEO_FRAME_DURATION;

        // successful iterations 
        static int count = 0;
        qDebug() << "count: " << count++;
    }

    RETURN_ON_BAD_HR(pSinkWriter->Finalize());

    MFShutdown();
    CoUninitialize();
}

Initialization routines are taken directly from Desktop Duplication Sample and Sink Writer Tutorial. Video format definition is also taken from the Sink Writer tutorial, with changes to frame resolution and VIDEO_INPUT_FORMAT = MFVideoFormat_ARGB32.

However, after going through this loop ~20 times (sometimes a bit more, sometimes a bit less), it fails on either mSinkWriter->WriteSample() or mDeskDupl->AcquireNextFrame() with the following HRESULT message:

The application made a call that is invalid. Either the parameters of the call or the state of some object was incorrect. Enable the D3D debug layer in order to see details via debug messages.

With D3D debug layer enabled, I see that my frames (I suppose those objects are my frames) are for some reason not released and just keep piling on:

D3D11 WARNING: Live Object at 0x0000006504FFC290, Refcount: 17. [ STATE_CREATION WARNING #0: UNKNOWN]

I'm fairly certain that HRESULT fails are coming from my GPU running out of memory (also confirmed by watching GPU memory usage via GPU-Z utility). However, I have no idea what causes this memory leak, because I'm releasing every resource that I'm allocating (SafeRelease implementation).

EDIT: Changed pointers to smart pointers and added macro definition.

prazuber
  • 1,352
  • 10
  • 26
  • Which line of code outputs the diagnostic message? What object is referenced? I also see, that you are using lots of `RETURN_ON_BAD_HR` macros, **before** calling `SafeRelease`. You tagged your question as [tag:c++], why not use a smart pointer instead (e.g. [CComPtr](https://msdn.microsoft.com/en-us/library/ezzw7k98.aspx) or [_com_ptr_t](https://msdn.microsoft.com/en-us/library/417w8b3b.aspx))? – IInspectable Jan 18 '16 at 15:02
  • @IInspectable I edited the lines that are problematic: those are SinkWriter's [`WriteSample()`](https://msdn.microsoft.com/en-us/library/windows/desktop/dd374654(v=vs.85).aspx) and sometimes Desk Dupl [`AcquireNextFrame()`](https://msdn.microsoft.com/en-us/library/windows/desktop/hh404615(v=vs.85).aspx). I agree that smart pointers are better, but under normal circumstances the macro shouldn't return anyway, so the memory leak isn't caused by it. As for the object that is referenced, I have no idea how to process D3D debug output. – prazuber Jan 18 '16 at 15:11
  • Smart pointers aren't *"better"*, they are **mandatory**, when using early-out code paths. You are potentially skipping resource cleanup. Since you have a resource leak, chances are, that you erroneously skipped cleanup code. Either go with the pattern used in the Sink Writer Tutorial, or use smart pointers. Since you aren't properly initializing pointers anyway, I would **strongly** suggest going with smart pointers. – IInspectable Jan 18 '16 at 15:28
  • @IInspectable Thanks for your advice, I changed my code to include smart pointers. However, my original question still stands - this loop still fails after 10 to 30 iterations and the memory is not freed. – prazuber Jan 18 '16 at 15:57
  • Your code is very wrong. An `HRESULT` designates success, when it is non-negative. Comparing an `HRESULT` against a single success code (`S_OK`) is asking for trouble. Use the [SUCCEEDED](https://msdn.microsoft.com/en-us/library/windows/desktop/ms687197.aspx) macro instead. Tearing down COM on the calling thread (`CoUninitialize`) in case a single COM call fails is pretty wrong, too. You need to understand sample code before you can safely change it. – IInspectable Jan 18 '16 at 16:10
  • @IInspectable Thank you once again, I rewrote my macro and added full thread code to the question. However, I can't put `CoUninitialize` in the destructor since the `run()` method is the thread entry point (I'm using `QThread` here), so I put those calls in my macro to correctly exit the thread. – prazuber Jan 18 '16 at 16:44
  • Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/100994/discussion-between-prazuber-and-iinspectable). – prazuber Jan 18 '16 at 16:47

0 Answers0