Running ffmpeg in a while loop causes memory leak

Question

I'm trying to run the following ffmpeg commands in a while loop synchronously.

Each time round the loop I wait for the current command to finish before I move on the next one. However I'm getting unexpected results and then it crashes.

  import ffmpegPath from '@ffmpeg-installer/ffmpeg'
  import ffmpeg from 'fluent-ffmpeg'

  ffmpeg.setFfmpegPath(ffmpegPath.path)

  const command = ffmpeg()

  let VIDEOS = 1000
  let videoIdx = 1

  while (videoIdx <= VIDEOS) {
    await new Promise((resolve) => {
      command
        .on('end', () => {
          setTimeout(() => {
            console.log(`${videoIdx}/${VIDEOS}`)
            videoIdx++
            resolve()
          }, 100)
        })
        .on('error', () => {
          console.log('error = ', error)
        })
        .input(`MyExernalHardDrive/input/video-${videoIdx}-frame-%d.png`)
        .inputFPS(1/0.0425)
        .output(`MyExernalHardDrive/output/video-${videoIdx}.mp4`)
        .outputFPS(24)
        .noAudio()
        .run()
    })
  }

In the console I'm expecting to see:

1/1000

then

2/1000

then

3/1000

etc..

Instead I'm getting logs in batches like:

1/1000
2/1000
3/1000

then

4/1000
5/1000
6/1000
7/1000

then it keeps incrementing but I get them in even bigger batches:

45/1000
46/1000
47/1000
48/1000
49/1000
50/1000
51/1000
52/1000
53/1000
54/1000
55/1000

And then I get:

(node:14509) MaxListenersExceededWarning: Possible EventEmitter memory leak detected. 
11 end listeners added to [FfmpegCommand]. 
Use emitter.setMaxListeners() to increase limit

I then exit the process and these are the files that have been created:

video-1.mp4
video-2.mp4
video-4.mp4
video-7.mp4
video-11.mp4
video-16.mp4
video-22.mp4
video-29.mp4
video-37.mp4
video-46.mp4
video-56.mp4
video-67.mp4

So it seems that:

The timing is all over the place and not sequential at all
Processes are piling up until there is a memory leak

Why is this happening since I'm using await before I move on to the next process and how do I fix this ?

Have you tried _not_ re-using the `command` variable in every loop iteration and using a new instance of `ffmpeg()`? I can't imagine reusing the same event emitter is the correct way to go. eg: `ffmpeg().on('end', ...).....run()`, rather than `command.on('end', ...)...run()`. — RickN, Dec 20 '21 at 16:37
Your suggestion solved the problem. You can turn it into an answer. :) Thanks once again — Kawd, Dec 20 '21 at 19:15
I've posted my comment as an answer. Sorry it's not more in-depth, I've tried to make it useful enough for others facing the same issue. — RickN, Dec 21 '21 at 10:58

score 2 · Accepted Answer · answered Dec 21 '21 at 10:55

// ...
  const command = ffmpeg()

  let VIDEOS = 1000
  let videoIdx = 1

  while (videoIdx <= VIDEOS) {
    await new Promise((resolve) => {
      command
        .on('end', () => {
          setTimeout(() => {
            console.log(`${videoIdx}/${VIDEOS}`)
            videoIdx++
            resolve()
          }, 100)
        })
// ...

The ffmpeg instance in variable command is being reused in every loop. It's an Event Emitter and shouldn't be reused, because some events only fire once, like "end". The "end" event on this single instance receives many listeners. The Node runtime is giving a warning, because this is usually not something you want (if it is, you can opt-in to disable the limit).

The solution is to not re-use the same instance:

// ...
  let VIDEOS = 1000
  let videoIdx = 1

  while (videoIdx <= VIDEOS) {
    await new Promise((resolve) => {
      ffmpeg()
        .on('end', () => {
          setTimeout(() => {
            console.log(`${videoIdx}/${VIDEOS}`)
            videoIdx++
            resolve()
          }, 100)
        })
// ...

Now every loop iteration a fresh instance with a clean slate will be created.

Running ffmpeg in a while loop causes memory leak

1 Answers1