2

I'm using ffmpeg to create a mosaic of videos using the xstack filter. The input videos may come in varying dimensions, so I'm using the scale filter to scale them beforehand, and I'm using the force_original_aspect_ratio option and then the pad filter to keep the original aspect ratios of each video and add black bars to the sides to make each video have the correct dimensions.

I have a command that's working - however, it's inconsistent. For some dimensions it works, while for others it doesn't.

I'm using the fluent-ffmpeg Node.js module to call ffmpeg from Node.js. To do this, I'm passing an array of strings to the complexFilter() function.

The following strings for the complex filter works:

"[0:v]scale=400:250:force_original_aspect_ratio=decrease,pad=400:250:(ow-iw)/2:(oh-ih)/2 [s0]"
"[1:v]scale=400:250:force_original_aspect_ratio=decrease,pad=400:250:(ow-iw)/2:(oh-ih)/2 [s1]"
"[s0][s1]xstack=inputs=2:fill='black':layout=0_0|w0_0[v]"
"[0:a][1:a]amix=inputs=2[a]"

However, if I change the output dimensions of each video to be 400:225 instead of 400:250 it fails.

"[0:v]scale=400:225:force_original_aspect_ratio=decrease,pad=400:225:(ow-iw)/2:(oh-ih)/2 [s0]"
"[1:v]scale=400:225:force_original_aspect_ratio=decrease,pad=400:225:(ow-iw)/2:(oh-ih)/2 [s1]"
"[s0][s1]xstack=inputs=2:fill='black':layout=0_0|w0_0[v]"
"[0:a][1:a]amix=inputs=2[a]"

The following error is given:

An error occurred: ffmpeg exited with code 1: Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #1:1
Conversion failed!

If it's relevant, the first video has dimensions of 1280x720 while the second video has dimensions of 320x240.

Anyone know why one set of dimensions works while the other doesn't?

EDIT: Here is the full ffmpeg log for when it fails:

ffmpeg version git-2020-05-13-b12b053 Copyright (c) 2000-2020 the FFmpeg developers
  built with gcc 9.3.1 (GCC) 20200513
  configuration: --enable-gpl --enable-version3 --enable-sdl2 --enable-fontconfig --enable-gnutls --enable-iconv --enable-libass --enable-libdav1d --enable-libbluray --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libsrt --enable-libtheora --enable-libtwolame --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libzimg --enable-lzma --enable-zlib --enable-gmp --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvo-amrwbenc --enable-libmysofa --enable-libspeex --enable-libxvid --enable-libaom --disable-w32threads --enable-libmfx --enable-ffnvcodec --enable-cuda-llvm --enable-cuvid --enable-d3d11va --enable-nvenc --enable-nvdec --enable-dxva2 --enable-avisynth --enable-libopenmpt --enable-amf
  libavutil      56. 45.100 / 56. 45.100
  libavcodec     58. 84.100 / 58. 84.100
  libavformat    58. 43.100 / 58. 43.100
  libavdevice    58.  9.103 / 58.  9.103
  libavfilter     7. 80.100 /  7. 80.100
  libswscale      5.  6.101 /  5.  6.101
  libswresample   3.  6.100 /  3.  6.100
  libpostproc    55.  6.100 / 55.  6.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:/Users/user_name/Desktop/vids/vid1.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp41isom
    creation_time   : 2020-05-21T15:52:20.000000Z
  Duration: 00:00:10.76, start: 0.000000, bitrate: 8385 kb/s
    Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc), 1280x720 [SAR 1:1 DAR 16:9], 8237 kb/s, 29.99 fps, 30 tbr, 30k tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2020-05-21T15:52:20.000000Z
      handler_name    : VideoHandler
      encoder         : AVC Coding
    Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 165 kb/s (default)
    Metadata:
      creation_time   : 2020-05-21T15:52:20.000000Z
      handler_name    : SoundHandler
Input #1, mov,mp4,m4a,3gp,3g2,mj2, from 'C:/Users/user_name/Desktop/vids/vid2.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp41isom
    creation_time   : 2020-05-21T15:54:37.000000Z
  Duration: 00:00:11.01, start: 0.000000, bitrate: 836 kb/s
    Stream #1:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc), 320x240 [SAR 1:1 DAR 4:3], 669 kb/s, 29.88 fps, 30 tbr, 30k tbn, 60 tbc (default)
    Metadata:
      creation_time   : 2020-05-21T15:54:37.000000Z
      handler_name    : VideoHandler
      encoder         : AVC Coding
    Stream #1:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 163 kb/s (default)
    Metadata:
      creation_time   : 2020-05-21T15:54:37.000000Z
      handler_name    : SoundHandler
Stream mapping:
  Stream #0:0 (h264) -> scale
  Stream #0:1 (aac) -> amix:input0
  Stream #1:0 (h264) -> scale
  Stream #1:1 (aac) -> amix:input1
  xstack -> Stream #0:0 (libx264)
  amix -> Stream #0:1 (aac)
Press [q] to stop, [?] for help
[swscaler @ 000001343fefc200] deprecated pixel format used, make sure you did set range correctly
[Parsed_pad_1 @ 000001343f8dc3c0] Padded dimensions cannot be smaller than input dimensions.
[Parsed_pad_1 @ 000001343f8dc3c0] Failed to configure input pad on Parsed_pad_1
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument
Error while processing the decoded data for stream #1:1
Conversion failed!

Done in 0.66s.
dfriend21
  • 89
  • 7

1 Answers1

0

I found a work-around, if not necessarily a solution.

The following message shows up in the log posted in the question.

Padded dimensions cannot be smaller than input dimensions.

I found that if I decrease each of the dimensions passed to scale by 1 (so that they are always less than the dimensions passed to pad), then I don't seem to get the error anymore.

The following complex filter strings work:

"[0:v]scale=399:224:force_original_aspect_ratio=decrease,pad=400:225:(ow-iw)/2:(oh-ih)/2 [s0]"
"[1:v]scale=399:224:force_original_aspect_ratio=decrease,pad=400:225:(ow-iw)/2:(oh-ih)/2 [s1]"
"[s0][s1]xstack=inputs=2:fill='black':layout=0_0|w0_0[v]"
"[0:a][1:a]amix=inputs=2[a]"

This isn't ideal, and I still don't understand why it wasn't working before. I would guess there's a more graceful way around this, but at least this works.

dfriend21
  • 89
  • 7