3

I would like to know if anyone could point me in the right direction here.

Suppose I have a video file in some encoding (e.g., H.264), and I decode this video using CUDA's video decoding library (https://developer.nvidia.com/nvidia-codec-libraries). What this would do (I presume), is it would decode the video and play it on the screen. So far so good.

Now, what I would really like to do is modify the decoding library (or write a wrapper, or something of that sort), to perform some post-processing on the video frames.

For example, suppose the video has 1000 frames, and I would like to (in real-time) add some custom effects to some (specified by the user) of the frames (e.g., render a 3d model---passed in by the user---and overlay the output to some location in the frame, etc).

I guess my question really is:

  1. Is this possible at all?
  2. Does anybody know of any code samples that can accomplish this or something similar (or tutorials, papers, manuals, anything at all)?

Thanks!

zb'
  • 8,071
  • 4
  • 41
  • 68
sga001
  • 158
  • 1
  • 9
  • You might also consider posting this on the Audio-Video Production StackExchange. Possible someone there hase more insight from the A/V angle (existing software solutions etc.) http://avp.stackexchange.com/ – jimbo Nov 21 '12 at 03:29

1 Answers1

1

It's certainly possible. The CUDA video decoding library will give you the video frame as a NV12-format matrix, which you can then do whatever you want with. There's nothing about it which will automatically display the decoded frame to the screen; you have to take care of that part yourself. If you're looking to have it run in real-time, then you should probably look into the CUDA <--> OpenGL interoperability, as you can map OpenGL buffer, texture, and renderbuffer objects into the address space of CUDA, so if you output your processed frame into a mapped object, you can display directly. I would advise looking at the CUDA samples cudaDecodeGL or cudaDecodeD3D9 (if you like DirectX instead of OpenGL).

Of course, getting what you're proposing to run in real-time is another matter...

alrikai
  • 4,123
  • 3
  • 24
  • 23