0

I have an IMFSampleGrabberSinkCallback in my Media Foundation application, which I create an activator for using MFCreateSampleGrabberSinkActivate. The function receives only a partially filled media type. Within the sample grabber callback, an NVENC encoder is running (I know that there are out-of-the-box transforms for NVENC, but the software is not running on Windows 10). In order to use the encoder, I need to have the full media type that after topology resolution (mainly the size of the frames, which is not known when the sample grabber is created).

There seems to be no obvious way to obtain this information in the IMFSampleGrabberSinkCallback, so the idea would be getting the full topology from the session once I receive MF_TOPOSTATUS_READY.

First question: Is this the only way to get the full type the grabber will receive or did I miss something?

After getting the full topology, the idea is iterating all nodes, searching the one with the sample grabber and retrieving its input type.

Second question: What would be the best way of doing that? For me, there seems to be no way to retrieve my IMFSampleGabberSinkCallback from the topology node.

I would solve that using the topo node ID, but I am not sure whether this would work in any case. The documentation of IMFTopologyNode::GetTopoNodeID states that

When a node is first created, it is assigned an identifier. Node identifiers are unique within a topology, but can be reused across several topologies. The topology loader uses the identifier to look up nodes in the previous topology, so that it can reuse objects from the previous topology.

To me, it is not clear whether the ID will be reused or whether it can be reused.

Third question: Is it guaranteed that, if I obtain the topo node ID of the node I insert my IMFActivate obtained from MFCreateSampleGrabberSinkActivate (and as long as I do not change the ID manually using IMFTopologyNode::SetTopoNodeID, the very same ID will be used for any subsequent topology created during the topology resolution process?

Thanks in advance, Christoph

Edit: The question is not on how to get NVENC working and resolve its input type, but the key questions are (1) whether the topo IDs are guaranteed to be preserved during the resolution process and (2) whether there is a better way to do that than the following (which is susceptible to users actively changing the topo ID): First, I create an activator for my grabber callback. Once I have added the node containing the activator as object to the topology, I retrieve its ID. When the session reports that the topology is ready, I retrieve the node with the ID saved before, obtain its object, query its IMFStreamSink interface, retrieve its IMFMediaTypeHandler, get the current media type and obtain the actual frame size and frame rate for NVENC. But: This only works if the ID cannot change and is not actively changed.

I have extracted the topology resolution stages from my test code:

Topology resolution stages

The topology resolver finds out that a colour conversion is required and adds the blue transform to account for this. Coming back to the questions: In this case, they would be (1) whether it is guaranteed that the ID of the red node cannot change during resolution and (2) whether there is an alternative way to implement this which is not susceptible to someone using IMFTopologyNode::SetTopoNodeID on the red node.

Christoph
  • 1,964
  • 2
  • 27
  • 44
  • Hi, Can you inform the correct reason for using of `IMFSampleGrabberSinkCallback`? It is general interfaces for grabbing of samples. It can be used for grabbing decoded UNCOMPRESSED images, or for compressed data - for example - JPEG frames from MJPG. Have you checked documentation of NVENC? – Evgeny Pereguda Aug 16 '16 at 10:47
  • I know that there is an `IMFTransform` from NVENC, but this only works on Windows 10+. I need to run on Windows 7+. Also, the code is working perfectly using a hard-coded frame size. I just need a way to obtain the media type dynamically. – Christoph Aug 16 '16 at 13:03
  • What type of MediaType do you need? Compressed or Uncompressed? - You have encoder NVENC - it needs two media type? – Evgeny Pereguda Aug 16 '16 at 13:24
  • I use NV12 as input, because this seems to be the only one NVENC accepts. However, the real question is how to get the full `IMFMediaType` that the topology resolver determined. – Christoph Aug 16 '16 at 20:23
  • Hi, I cannot give all answers on your questions, but I can recommend use `IMFTopology::GetOutputNodeCollection` after resolving topology - it allows get output node with `IMFSampleGabberSinkCallback` without iterating of all nodes. However, I cannot correct understand what type of `IMFMediaType` do you need? I mean - for encoder NVENC it needs set input uncompressed MediaType and Compressed MediaType. However, for some encoders it can need only set input uncompressed MediaType - and get the suitable Compressed MediaType via `IMFTransform::GetOutputAvailableType . – Evgeny Pereguda Aug 16 '16 at 22:53
  • For example Microsoft encoder H264 after setting of uncompressed MediaType it needs to get two Compressed MediaTypes via `IMFTransform::GetOu‌​tputAvailableType` - first for Baseline Profile and second for High profile, and set one of them into `IMFTransform::SetOu‌​tputType`. The fact is that Compressed MediaType usually has additional meta info, which must be set by encoder. I has some experience with setting of encoder from my project - [CaptureManager SDK](http://www.codeproject.com/Articles/1017223/CaptureManager-SDK), but I cannot understand clear - what do you need? – Evgeny Pereguda Aug 16 '16 at 23:00
  • Basically, all of the things I described in the question *work*, but the key question is whether that ID-based "hack" will *always* work, ie whether there are any guarantees that the topology resolver will not change the IDs of the nodes and that the nodes themselves never change during the process. – Christoph Aug 17 '16 at 09:03
  • What I do is: assign an NV12-based media type to my grabber, obtain the TOPOID of the node created for it, let the resolver do its work, and then, I want to get the resolved input type of my grabber node, because the resolver has propagated the full media type to this node. I have everything I need and the encoder works perfectly, only the width and height of the resolved media type is missing. – Christoph Aug 17 '16 at 09:06
  • I see your need and I can recommend use `MFCreateVideoMediaTypeFromBitMapInfoHeaderEx` and `Color Converter DSP`. The first - `Creates a video media type from a BITMAPINFOHEADER structure.`. The second [Color Converter DSP](https://msdn.microsoft.com/en-us/library/windows/desktop/ff819079(v=vs.85).aspx) allows convert from RGB32 into NV12 or other formats. – Evgeny Pereguda Aug 17 '16 at 11:45
  • On page [CColorConvertDMO](http://stackoverflow.com/questions/37461426/windows-media-foundation-using-imftransform-to-decode-mp4-movie-frames-to-2d-tex/37472309?noredirect=1#comment62617956_37472309) you can find code for using of CColorConvertDMO. If you use video from file - you can get the width and height from MediaSource file. – Evgeny Pereguda Aug 17 '16 at 11:47
  • The image formats and colour conversion are really not the issue - I only mentioned NVENC to illustrate what I am doing. I have edited my post to highlight what I already tried/what is working and what I still need to know. – Christoph Aug 18 '16 at 08:35
  • 1
    Hi, I see the reason of my misunderstanding. From my experience ID of Topology Node looks like result of hash function. I think that generator of ID has complex algorithm and it cannot guaranty constant from one session to other, but iID is stable during current session. Maybe you can try another way - your wrote - "have added the node containing the activator as object to the topology,, " but what do you do with original pointer on "an activator for my grabber callback"? `IMFTopologyNode::SetObject` increments reference of `IUnknown`. I think you release original pointer an activate, but you – Evgeny Pereguda Aug 18 '16 at 09:05
  • 1
    can keep pointer on on "an activator for my grabber callback". In that case there is not need - "I retrieve the node with the ID saved before, obtain its object,". After resolving of topology you ALREADY can have pointer on activator with full resolved MediaType and query its IMFStreamSink interface, retrieve its IMFMediaTypeHandler without saving ID of topology node. – Evgeny Pereguda Aug 18 '16 at 09:08
  • But in this case, I would have the pointer of the activator, not of the actual `IMFStreamSink`? I must admin that I did not completely understand the concept of the activators, but in my understanding, this is some kind of proxy object which creates the (in my example) `IMFStreamSink` during the topology resolution process. Is that correct? – Christoph Aug 18 '16 at 09:11
  • Yes - activator is proxy, but for `IMFMediaSink` object, which is gotten by calling of `IMFActivate::ActivateObject` with IID_IMFMediaSink. While it is called first time by topology resolving it create object with `IMFMediaSink` interface (which creates the `IMFStreamSink` with `IMFSampleGabberSinkCallback` inner itself) in activator, but the next calling will return reference - Activator KEEP reference on resolved `IMFMediaSink`. Object of `IMFStreamSink` is got by `IMFMediaSink::GetStreamSinkByIndex ` – Evgeny Pereguda Aug 18 '16 at 09:23
  • I'm not sure whether I understand you correctly: You mean I save the activator, wait until the topology has been resolved, and then, the activator will give me the very same object that it created when it was called the first time? – Christoph Aug 18 '16 at 09:32
  • 1
    Yes. Theoretically your a right according to MSDN [MFActivate::ActivateObject](https://msdn.microsoft.com/en-us/library/windows/desktop/ms694292(v=vs.85).aspx) - "After the first call to ActivateObject, subsequent calls return a pointer to the same instance, until the client calls either ShutdownObject or IMFActivate::DetachObject." Of course, after topology resolving inner code of Microsoft can call `IMFActivate::DetachObject`, but you can check it - I think that it does not detached MediaSink object from activator. – Evgeny Pereguda Aug 18 '16 at 09:39
  • If you posted an answer, I would accept it - it's working ... – Christoph Aug 20 '16 at 16:37

1 Answers1

0

From my experience ID of Topology Node looks like result of hash function. I think that generator of ID has complex algorithm and it cannot guaranty constant from one session to other, but ID is stable during current session. Maybe you can try another way - your wrote - "have added the node containing the activator as object to the topology," but what do you do with original pointer on "an activator for my grabber callback"? IMFTopologyNode::SetObject increments reference of IUnknown. I think you release original pointer an activate, but you
can keep pointer on on "an activator for my grabber callback". In that case there is not need - "I retrieve the node with the ID saved before, obtain its object,". After resolving of topology you ALREADY can have pointer on activator with full resolved MediaType and query its IMFStreamSink interface, retrieve its IMFMediaTypeHandler without saving ID of topology node. Activator is proxy, but for IMFMediaSink object, which is gotten by calling of IMFActivate::ActivateObject with IID_IMFMediaSink. While it is called first time by topology resolving it creates object with IMFMediaSink interface (which creates the IMFStreamSink with IMFSampleGabberSinkCallback inner itself) in activator, but the next calling will return reference - Activator KEEPS reference on resolved IMFMediaSink - according to MSDN MFActivate::ActivateObject - "After the first call to ActivateObject, subsequent calls return a pointer to the same instance, until the client calls either ShutdownObject or IMFActivate::DetachObject." It means that after resolving of topology Microsoft's code DOES NOT detach object or shutdown it - only close session execute ShutdownObject or IMFActivate::DetachObject.

Regards

Evgeny Pereguda
  • 553
  • 1
  • 4
  • 9