tl;dr The sample I linked takes 2 minutes to generate thumbnails for a 2 minute video and I can't figure out how to speed it up without dropping frames which messes up what I'm trying to do.
I'm working on optimizing a .NET Core app that processes a large amount of files. I have previously run ffmpeg[.exe] out of process to generate thumbnail strips (multiple thumbnails from different points in the video) from videos as one of the steps, but after optimizations to everything else it's the slowest part of the process. For example on a 2 minute video it currently takes about 10 seconds. I am looking to move it in-process to speed it up.
I am trying to use libVLCSharp to do this and the sample here seems to be close to what I want to do:
I modified it to produce the images I need, the problem is this code plays the video at 1x speed. So my test video which is 2 minutes long takes 2 minutes to generate a thumbnail strip.
I tried using MediaPlayer.SetRate to speed up the playback (I tried 100 as a test, then float.PositiveInfinity to max it out) which at first seemed to work. The processing time is cut to 6 seconds. However VLC drops most of the frames in the process (the video callbacks never get them) so I can't use them, and most of the ones I can capture are duplicates of previous frames making the thumbnail strips I do get fairly useless (I only get 2 or 3 unique thumbnails which repeat instead of, say, 8).
I can set the rate to a less extreme number like 2 and it works fine. But that doesn't help me much since I'm looking to get it under 10 seconds to be worth changing my existing code.
The next thing I want to try is find a way to disable the frame drop to see how fast that goes, but I can't find an API to adjust the frame drop.
So any help on how to do that would be appreciated. I've searched around and either I'm just calling it by the wrong name and can't find it, or libVLC does not expose that mechanism.
The other approach I was thinking of was just to jump to specific intervals in the video file and dump a single frame from each. I tried using MediaPlayer.Position to do this. Due to the way some videos are encoded I'm not sure if this will get me usable thumbnails but I can't even get it to work on a basic level. The video callbacks don't seem to get called even if I call .Play before changing Position, and MediaPlayer.TakeSnapshot seems to fail as well (though I would rather not use that since I want to build my own thumbnail strip rather than have individual images). I tried also calling MediaPlayer.NextFrame() after changing Position to try and force the video callbacks but that didn't do anything.
Any suggestions on how to get this working or alternative possible solutions would be welcome. I am not really interested in anything that runs out of process though since I am already dealing with that overhead in my current implementation so it wouldn't be worth changing to another one.
Also if I'm crazy and the existing sample doesn't actually have this problem let me know, but I'm following it pretty closely apart from the image generation and I don't see how it would be different.
Edit: I decided to try making minimal modifications to the sample to .Play and then set .Position to the location of the first frame. Then I wait until I have moved beyond that position (eg the video has finishing seeking and is playing again) and grab that frame for processing. After I have fully processed it outside of the video callbacks I advance .Position to the next frame I want if I'm not already there and so on and so forth. This seems to work and gets me a time comparable to my infinite SetRate. I do need to set .Position to 1 after I get all my frames though; calling player.Stop seems to hang for some reason and doing nothing after I'm done but disposing the Media and MediaPlayer seems to cause the app to wait around. Whatever, it works now. When libVLC 4 comes out I'll reevaluate.
Edit2: I have to let the video run to the end before I dispose MediaPlayer or it crashes my app. Fun.
