How to Concatenate bunch of videos using python?

Question

So, I have over 5000 small clips that I need to combine. To apply various custom filter over their names, I want to do it with python. I have the following code:

import os
from moviepy.editor import *
os.chdir('D:/videos')
list1, list2 = os.listdir(), []

for i in list1: #filtering
    if i[-6:] != '-l.mp4' and i[-7:] != 'ALT.mp4':
        list2.append(i)
print('Getting Video Info:')

final = VideoFileClip(list2[0])


for i in range(1,len(list2)):
    final = concatenate_videoclips([final, VideoFileClip(list2[i])])
    print('\r' + str(i+1) + '/' + str(len(list2)), end='')


os.chdir('D:')
final.write_videofile('Merged.mp4')

But the program is creating lots of processes and just after reading 150 clips it's crashing. It keeps increasing! Is there any easier way/alternative to do this? Thanks!

Edit:
I've tried using ffmpeg too, but concatenation removes the audio since concat protocol doesn't support .mp4 extension. In that case. Even if I convert all the files to .ts extension and try to concatenate them,WindowsError: [Error 206] The filename or extension is too long pops up because too many files are separated by |. I did the following changes after converting all the files to .ts format:

import os
import ffmpeg
os.chdir('D:/videos')
list1 = os.listdir()
list2 = [i for i in list1 if i[-3:] == '.ts']
list2[0] = ffmpeg.input(list2[0])
for i in range(1, len(list2)):
    list2[i] = ffmpeg.concat(list2[i-1], ffmpeg.input(list2[i]))
    print('\r' + str(i) + '/' + str(len(list2)), end='')
ffmpeg.output(list2[-1], 'D:\Merged.mp4')
ffmpeg.run(list2[-1])

But now I'm getting RecursionError: maximum recursion depth exceeded while calling a Python object.

If you're just asking for recommendations on alternative ways to combine mp4 video fragments, I don't think StackOverflow is the right place. As for your current solution, have you looked at what exactly causes it to crash? Does it run out of memory (which seems likely)? You might want to look at solutions that don't try to do this in memory, but operate on the files - or perhaps simply drive something like ffmpeg from a Python script? As long as the fragments are already compatible that should be straightforward and otherwise converting them first would be as well? — Grismar, Jan 17 '22 at 07:15
@Gismar Yes sir, it runs out of memory. So, [ffmpeg-python](https://github.com/kkroening/ffmpeg-python) operates on the files, right? — Saikat Chakraborty, Jan 17 '22 at 08:37

Alok Nayak · Answer 1 · 2022-01-17T14:31:08.933

Can you try deleting explicitly and garbage collection in between like this.

import os
import gc
from moviepy.editor import *
os.chdir('D:/videos')
list1, list2 = os.listdir(), []

for i in list1: #filtering
    if i[-6:] != '-l.mp4' and i[-7:] != 'ALT.mp4':
        list2.append(i)
print('Getting Video Info:')

final = VideoFileClip(list2[0])


for i in range(1,len(list2)):
    curclip = VideoFileClip(list2[i])
    final = concatenate_videoclips([final, curclip])
    print('\r' + str(i+1) + '/' + str(len(list2)), end='')
    curclip.close()
    del curclip
    gc.collect()

os.chdir('D:')
final.write_videofile('Merged.mp4')

If above doesn't work, try saving final video at a certain frequency e.g 8. Saving and reading as raw file will release extra memory accumulating inside final object

for i in range(1,len(list2)):
    if (i % 8) == 7:
        final.write_videofile('D:/Merged.mp4')
        final = VideoFileClip('D:/Merged.mp4')
    curclip = VideoFileClip(list2[i])
    final = concatenate_videoclips([final, curclip])
    print('\r' + str(i+1) + '/' + str(len(list2)), end='')
    curclip.close()
    del curclip
    gc.collect()

os.chdir('D:')
final.write_videofile('Merged.mp4')

It's a good hack, but as the filesize increases with each iteration of `final.write_videofile`, moviepy takes more and more time. It'll take way less time if I could write all the videos at once, right? — Saikat Chakraborty, Jan 17 '22 at 11:32
Basically final should be cleaned some how but I am not able to find suitable implementation in the moviepie repo: https://github.com/Zulko/moviepy/blob/master/moviepy/video/VideoClip.py .Closing videofileclip object also looks like a good practise. — Alok Nayak, Jan 17 '22 at 14:36

How to Concatenate bunch of videos using python?

1 Answers1