1

I have to burn subtitles based image on video using ffmpeg(v4.3) and cuda, hardware accelerator. I want to use 'overlay_cuda' in filter complex.

It is my command.

./ffmpeg -init_hw_device cuda=cuda:1 -hwaccel cuda -filter_hw_device cuda -hwaccel_output_format cuda -i 2_dump.ts -filter_complex "[v:0]scale_npp=1920:1080,format=yuv420p[base_video];[base_video][s:3]overlay_cuda[resurlt_video]" -map "[resurlt_video]" -map 0:a -c:v h264_nvenc -f mpegts subtitle_test.ts

But it's fail.

These are output massage included error.

ffmpeg version 4.3 Copyright (c) 2000-2020 the FFmpeg developers
built with gcc 4.8.5 (GCC) 20150623 (Red Hat 4.8.5-39)
configuration: --prefix=/usr/local --pkg-config-flags=--static --extra-cflags=-I/usr/local/include --extra-ldflags='-L/usr/local/lib -L/usr/local/lib64' --extra-libs='-lm -lpthread -lc' --bindir=/usr/local/bin --enable-cross-compile --enable-pic --enable-gpl --enable-libfdk_aac --enable-libfreetype --enable-libmp3lame --enable-libopus --enable-libvpx --enable-libx264 --enable-libx265 --enable-nonfree --enable-cuda-nvcc --enable-cuvid --enable-nvenc --enable-libnpp --extra-cflags=-I/usr/local/cuda/include --extra-ldflags=-L/usr/local/cuda/lib64 --enable-libass --enable-static --disable-shared
libavutil      56. 51.100 / 56. 51.100
libavcodec     58. 91.100 / 58. 91.100
libavformat    58. 45.100 / 58. 45.100
libavdevice    58. 10.100 / 58. 10.100
libavfilter     7. 85.100 /  7. 85.100
libswscale      5.  7.100 /  5.  7.100
libswresample   3.  7.100 /  3.  7.100
libpostproc    55.  7.100 / 55.  7.100
[mpegts @ 0x4971500] start time for stream 5 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 6 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 7 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 8 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 9 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 10 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 11 is not set in estimate_timings_from_pts
[mpegts @ 0x4971500] start time for stream 12 is not set in estimate_timings_from_pts
Input #0, mpegts, from '2_dump.ts':
Duration: 00:05:09.80, start: 1.400000, bitrate: 12595 kb/s
Program 1 
Metadata:
  service_name    : Service01
  service_provider: AMUZLAB
Stream #0:0[0x100](eng): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, 5.1(side), fltp, 384 kb/s
Stream #0:1[0x101](ind): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:2[0x102](zho): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:3[0x103](kho): Audio: ac3 ([129][0][0][0] / 0x0081), 48000 Hz, stereo, fltp, 192 kb/s
Stream #0:4[0x104]: Video: h264 (High) ([27][0][0][0] / 0x001B), yuv420p(top first), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 50 tbr, 90k tbn, 50 tbc
Stream #0:5[0x105](CHI): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:6[0x106](CHS): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:7[0x107](IND): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:8[0x108](THA): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:9[0x109](MAN): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:10[0x10a](MON): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:11[0x10b](BUR): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
Stream #0:12[0x10c](ENG): Subtitle: dvb_subtitle ([6][0][0][0] / 0x0006)
File 'subtitle_test.ts' already exists. Overwrite? [y/N] y
Stream mapping:
Stream #0:4 (h264) -> scale_npp (graph 0)
Stream #0:8 (dvbsub) -> overlay_cuda:overlay (graph 0)
overlay_cuda (graph 0) -> Stream #0:0 (h264_nvenc)
Stream #0:0 -> #0:1 (ac3 (native) -> mp2 (native))
Stream #0:1 -> #0:2 (ac3 (native) -> mp2 (native))
Stream #0:2 -> #0:3 (ac3 (native) -> mp2 (native))
Stream #0:3 -> #0:4 (ac3 (native) -> mp2 (native))
Press [q] to stop, [?] for help
[mpegts @ 0x4971500] sub2video: using 1920x1080 canvas
Impossible to convert between the formats supported by the filter 'Parsed_scale_npp_0' and the filter 'auto_scaler_0'
Error reinitializing filters!
Failed to inject frame into filter network: Function not implemented
Error while processing the decoded data for stream #0:4
Conversion failed!

And, It is command line succeeded used 'overlay', software video filter, non_hardware accelerator.

./ffmpeg -hwaccel cuda -re -i 2_dump.ts -filter_complex "[0:v:0][0:s:3]overlay[v]" -map "[v]" -map 0:a -c:v h264_nvenc -f mpegts subtitle_test.ts

How to to burn subtitles based image on video using ffmpeg(v4.3) and cuda, hardware accelerator?

Thank you.

jgkim0518
  • 39
  • 3
  • Use "hwupload_cuda" before scale_npp, followed by "hwdownload" ? – Prabindh Sep 04 '20 at 03:44
  • /./ffmpeg -init_hw_device cuda=cuda:1 -hwaccel cuda -filter_hw_device cuda -hwaccel_output_format cuda -i 2_dump.ts -filter_complex "[v:0]hwupload_cuda,scale_npp=1920:1080,format=yuv420p,hwdownload[base_video];[base_video][s:3]overlay_cuda[resurlt_video]" -map "[resurlt_video]" -map 0:a -c:v h264_nvenc -f mpegts subtitle_test.ts' right? – jgkim0518 Sep 04 '20 at 03:50
  • it makes this message. '[mpegts @ 0x5570540] sub2video: using 1920x1080 canvas Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:4' and the filter 'auto_scaler_0' / Error reinitializing filters! / Failed to inject frame into filter network: Function not implemented / Error while processing the decoded data for stream #0:4 / Conversion failed!' – jgkim0518 Sep 04 '20 at 03:51
  • hwdownload will bring it back to CPU, so please check if it is needed for your case, since you are using h264_nvenc – Prabindh Sep 04 '20 at 03:56
  • i think so. 'hwdownload' is needless. './ffmpeg -init_hw_device cuda=cuda:1 -hwaccel cuda -filter_hw_device cuda -hwaccel_output_format cuda -i 2_dump.ts -filter_complex "[0:v:0]hwupload_cuda,scale_npp=1920:1080,format=yuv420p[base_video];[base_video][0:s:3] overlay_cuda [resurlt_video]" -map "[resurlt_video]" -map 0:a -c:v h264_nvenc -f mpegts subtitle_test.ts' – jgkim0518 Sep 04 '20 at 04:06
  • it makes this message. [mpegts @ 0x3b49580] sub2video: using 1920x1080 canvas Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:4' and the filter 'auto_scaler_0' / Error reinitializing filters! / Failed to inject frame into filter network: Function not implemented / Error while processing the decoded data for stream #0:4 \ Conversion failed!' – jgkim0518 Sep 04 '20 at 04:06
  • I ran '-init_hw_device cuda=cuda:1 -hwaccel cuda -hwaccel_output_format cuda -filter_hw_device cuda -i 2_dump.ts -filter_complex "[0:v:0]scale_npp=1920:1080:format=nv12[base_video];[0:s:3]scale_npp=1920:1080:format=nv12[sub];[base_video]sub]overlay_cuda[resurlt_video]" -map "[resurlt_video]" -map 0:a -c:v h264_nvenc -f mpegts subtitle_test.ts' – jgkim0518 Sep 04 '20 at 07:45
  • But it make this maessage '[mpegts @ 0x3dd2580] sub2video: using 1920x1080 canvas Impossible to convert between the formats supported by the filter 'graph 0 input from stream 0:8' and the filter 'auto_scaler_0' Error reinitializing filters! Failed to inject frame into filter network: Function not implemented Error while processing the decoded data for stream #0:4'. too. why...why...help me please... – jgkim0518 Sep 04 '20 at 07:45
  • Does this help ? https://stackoverflow.com/questions/63471028/how-to-use-the-ffmpeg-overlay-cuda-filter-to-make-a-sbs-video – Prabindh Sep 04 '20 at 07:48
  • My command line run used two video input. But I want to make to burn subtitles on video. Like '-filter_complex "[0:v:0]hwupload_cuda,scale_npp=1920:1080,format=yuv420p[base_video];[base_video][0:s:3] overlay_cuda [resurlt_video]" -map "[resurlt_video]"' – jgkim0518 Sep 04 '20 at 08:18

1 Answers1

1

You can do like below,

ffmpeg -y -hwaccel cuvid -hwaccel_output_format cuda -i c:\outfile.mp4 -i C:\overlay.png -filter_complex "[0:v]scale_cuda=1280:720,format=cuda [base]; [1:v]format=nv12,hwupload_cuda,scale_cuda=1280:720,format=cuda [ovly1]; [base][ovly1] overlay_cuda,scale_cuda=1280:720,hwdownload,format=nv12[marked]" -map "[marked]" -vcodec h264_nvenc -map 0:1 -acodec copy testfile.mp4

In particular, the overlay_cuda only supports the below overlay formats:

NV12,YUV420P,YUVA420P

From https://github.com/FFmpeg/FFmpeg/blob/4976b102d89787d59a3aaa438b76c62aec86ad5a/libavfilter/vf_overlay_cuda.c

Prabindh
  • 3,356
  • 2
  • 23
  • 25