I'll start off by saying that this question is very similar to this other one - the main difference will be, I hope, I will provide enough context to be able to resolve the issue :)
I am writing a C++ library for a proprietary UVC camera device that utilizes ms media foundation to grab frames for processing and display through a custom C++/CUDA/C# application. The camera is occasionally attached and detached from the device during normal use, and at a rate of about 15% of the time, this will happen:
After a successful creation of an IMFSourceReader object, a subsequent asynchronous call to ReadSample is made. The callback function is first called with "hrStatus" set to S_OK, and the dwStreamFlags set to indicated MF_SOURCE_READERF_STREAMTICK. After this, any subsequent calls to the ReadSample function do not elicit a call to the registered OnReadSample callback.
Included below are snippets of the SourceReader initialization code, the call to ReadSample, and the OnReadSample implementation.
The init code for the Source Reader object
// Create the media source
res = SUCCEEDED(CreateVideoDeviceSource(&cap_mediasrc_des));
if (!res)
{
std::cout << "Failed to create video device source" << std::endl;
}
// Get the stream and presentation descriptors to set data type
.
. excluded this code for brevity
.
// Create source reader object and get handle to it
if (res)
{
// Create attributes store for the source reader
// to indicate that it should be used asynchronously
HRESULT hr = S_OK;
IMFAttributes *src_rdr_attr = NULL;
res = SUCCEEDED(hr = MFCreateAttributes(&src_rdr_attr, 1));
if (res)
{
res = SUCCEEDED(hr = src_rdr_attr->SetUnknown(MF_SOURCE_READER_ASYNC_CALLBACK, &frame_src_reader_cb));
}
if (res)
{
res = SUCCEEDED(MFCreateSourceReaderFromMediaSource(cap_mediasrc_des, src_rdr_attr, &cap_src_rdr_h));
if (!res)
{
std::cout << "Failed to create source reader from media source" << std::endl;
}
}
}
The call to ReadSample - made in a loop that runs continuously
res = SUCCEEDED(capture_result = cap_src_rdr_h->ReadSample(
MF_SOURCE_READER_FIRST_VIDEO_STREAM,
0,
NULL,
NULL,
NULL,
NULL
));
if (res)
{
// wait for async call to finish or to timeout, whichever comes first
std::uint32_t timeout_cnt = 0U;
while (!read_sample_available && (timeout_cnt < 100U))
{
Sleep(1);
timeout_cnt++;
}
if (timeout_cnt >= 100U)
{
read_time_expired = true;
}
}
and finally, the implementation of OnReadSample (with some image processing code excluded for brevity)
HRESULT FrameSource::FrameSourceReaderCallback::OnReadSample(
HRESULT hrStatus,
DWORD dwStreamIndex,
DWORD dwStreamFlags,
LONGLONG llTimestamp,
IMFSample *pSample
)
{
HRESULT res = hrStatus;
// Check the flags
if (((dwStreamFlags & MF_SOURCE_READERF_STREAMTICK) != MF_SOURCE_READERF_STREAMTICK) && (dwStreamFlags > 0))
{
// If the flags indicate something other than just MF_SOURCE_READERF_STREAMTICK, we've run into a problem
// Retain hrStatus if possible
res = SUCCEEDED(hrStatus) ? S_FALSE : hrStatus;
FrameSource::GetInstance().read_sample_available = false;
}
else if ((dwStreamFlags & MF_SOURCE_READERF_STREAMTICK) == MF_SOURCE_READERF_STREAMTICK)
{
// There isn't a new frame to grab
FrameSource::GetInstance().read_sample_available = false;
}
}
else if (SUCCEEDED(res))
{
// Good frame available
IMFMediaBuffer* in_buff_h = nullptr;
IMF2DBuffer2* in_buff2d_h = nullptr;
IMFMediaBuffer* out_buff_h = nullptr;
IMF2DBuffer2* out_buff2d_h = nullptr;
... here we convert the sample to a contiguous buffer, read some data out of it,
... run it through a color conversion MFT, and copy it into a CUDA GPU resident buffer
... excluded for brevity
if (SUCCEEDED(res))
{
frm_src_ref.read_sample_available = true;
}
// Make sure to release everything that we allocated
if (in_buff_h != nullptr)
{
in_buff_h->Release();
}
if (in_buff2d_h != nullptr)
{
in_buff2d_h->Release();
}
if (out_buff_h != nullptr)
{
out_buff_h->Release();
}
if (out_buff2d_h != nullptr)
{
out_buff2d_h->Release();
}
}
else
{
// res failed, but no flags?
std::cout << "OnReadSample failed, but there are no error flags" << std::endl;
}
return res;
}
I am stumped and have been stumped for quite awhile. I would assume if there was some issue on the UVC side of things that the first ReadSample call would indicate this and I'd be able to diagnose this issue simply. Originally, I had a synchronous implementation of ReadSample that would run into a problem that presented the same way (first call gave MF_SOURCE_READERF_STREAMTICK, the next never returned). I am left wondering if there is some other media foundation way to check the status of a media source so that I might be able to debug this further?
What do you folks think? I am running Windows 10 Pro 64-Bit (10.0, build 18363) for reference. Any and all help is very appreciated. Thanks!