0

Edit: In this specific example, Python 3.8 is being used on Windows 10.

I need to run FFprobe through subprocess.Popen so I can capture and parse the output. If I do this in a script, it runs almost instantly, but if I compile my script with PyInstaller and do the same thing, it takes over 0.8 seconds to complete. Is there a trick or any way I can execute it faster from the compiled script? I'm willing to hear FFprobe-specific answers if they exist.

This is what I'm using currently. The extra code is for ensuring FFprobe doesn't flash a console window while running (removing the console window actually made it 0.1 seconds faster on average).

startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
p = subprocess.Popen(f'ffprobe -show_format -show_streams -of json "{file}"', startupinfo=startupinfo, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
out, err = p.communicate()
thisismy-stackoverflow
  • 343
  • 2
  • 9
  • On which platform? `fork()` is inherently expensive on Windows -- unixlikes are optimized for `fork()`+`exec()` as the way programs get started and so it's very cheap there, whereas Windows has a `spawn()` call that processes are expected to use because it can't do fork well. – Charles Duffy May 06 '22 at 17:47
  • @CharlesDuffy It needs to be cross-platform, but in this specific example I was testing it on Windows. Is there no way I can get around this on Windows? Even with a hacky solution? – thisismy-stackoverflow May 06 '22 at 17:50
  • ...and a pyinstaller image, yes, is going to have a big memory footprint and be _particularly_ expensive to fork. – Charles Duffy May 06 '22 at 17:50
  • There might be a hacky solution, but I'm the wrong person to know what it is -- I'm a UNIX developer, so my answer is "be glad it's not my problem". :) – Charles Duffy May 06 '22 at 17:50
  • ...it does look like new enough versions of the Python runtime (3.8 and later?) support using `spawn()` syscalls instead of `fork()+exec()` on UNIXlikes, which I suppose implies they _might_ have that same improvement on Windows... which version of Python are you testing with? – Charles Duffy May 06 '22 at 17:52
  • @CharlesDuffy Python 3.8, specifically because I thought it was funny that using 3.8 gave me backwards compatibility all the way back to Windows Vista without having to change anything in my project. Does 3.8 support that? Edit: nevermind I see you mentioned it. I'll check it out – thisismy-stackoverflow May 06 '22 at 17:55
  • If you had been using a version of Python older than 3.8, I was suggesting to check if you still had the problem on 3.8. Sounds like the answer is "yes". – Charles Duffy May 06 '22 at 18:00
  • @CharlesDuffy You're probably not interested, but I "solved" this by adding ">" to the command to redirect FFprobe's output to a text file, then opening and parsing said text file. It ended up being ~4.5x faster than directly communicating with Popen, is cross-platform (I think, haven't tested macOS), gives the added benefit of being able to reuse the text file, and is just _barely_ fast enough for my use-case. – thisismy-stackoverflow May 12 '22 at 18:07

0 Answers0