Gstreamer - stream with image overlay to youtube

Question

trying to stream from my Jetson nano with picamera 2 to youtube with gstreamer.
Streaming only video works, but i need to overlay video with image using multifilesrc(image will change over time). After many hours a was not sucesfull to incorporate multifilesrc into pipeline. I have tried compositor, videomixer but all failed. Maybe using nvcompositor? Any ideas?

This is what i have so far

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! \
  "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1" ! omxh264enc ! \
  'video/x-h264, stream-format=(string)byte-stream' ! \
   h264parse ! queue ! flvmux name=muxer alsasrc device=hw:1 ! \
   audioresample ! "audio/x-raw,rate=48000" ! queue ! \
   voaacenc bitrate=32000 ! aacparse ! queue ! muxer. muxer. ! \
   rtmpsink location="rtmp://a.rtmp.youtube.com/live2/x/xxx app=live2"

EDIT: tried this but not working

gst-launch-1.0 \
    nvcompositor name=mix sink_0::zorder=1 sink_1::alpha=1.0 sink_1::zorder=2 ! nvvidconv ! omxh264enc ! \
    'video/x-h264, stream-format=(string)byte-stream' ! \
    h264parse ! queue ! flvmux name=muxer alsasrc device=hw:1 ! \
    audioresample ! "audio/x-raw,rate=48000" ! queue ! \
    voaacenc bitrate=32000 ! aacparse ! queue ! muxer. muxer. ! \
    rtmpsink location="rtmp://a.rtmp.youtube.com/live2/x/xxx app=live2" \
    nvarguscamerasrc sensor-id=0 ! \
    "video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)30/1" ! \
    nvvidconv ! video/x-raw, format=RGBA, width=1920, height=1080, framerate=30/1 ! autovideoconvert ! queue ! mix.sink_0 \
    filesrc location=logo.png ! pngdec ! alphacolor ! video/x-raw,format=RGBA ! imagefreeze ! nvvidconv ! mix.sink_1

SeB · Answer 1 · 2022-04-03T22:01:05.270

Although it may work in some cases without these, for using nvcompositor, I'd advise to use RGBA format in NVMM memory with pixel-aspect-ratio=1/1 for both inputs and for output. Use caps after nvvidconv for being sure in inputs pipelines, and use nvvidconv for converting nvcompositor output into NV12 (still in NVMM memory) before encoding.

You may also add a queue on 2nd input for logo before compositor. Probably not mandatory, but safer. You may also set a framerate in caps after imagefreeze.

Last, you may have to set xpos,ypos,width and height for all sources for a more reliable behavior.

Gstreamer - stream with image overlay to youtube

1 Answers1