Unable to open the output video file of my pipeline when using appsrc

Question

I'm creating a GStreamer pipeline in Python so that I stack consecutive camera images onto a buffer and pipe them to create a video (format does not matter). Each image in the buffer is encoded as a numpy array with all three RGB channels. I source the pipeline via GstAppSrc, convert it, encode it, mux it with appropiate caps and then save it via filesink. The code runs, but the actual video file is either empty or does not open in VLC or QuickTime Player, regardless of the format. Below is the code

class VideoStream():

    def __init__(self):
        self.buffer_size = 10
        self.width = 448
        self.height = 256
        self.buffer = np.zeros(
            (self.height, self.width, 3, self.buffer_size), dtype='uint8')
        # Height, Width, Channels (3 for rgb), buffer size
        self.index = 0
    
        Gst.init(sys.argv[1:])
        
                
        #Declare elements
        self.appsrc = Gst.ElementFactory.make("appsrc", "source")
        self.appsrc.set_property("is_live", True)
        self.appsrc.set_property("emit_signals", True)
        
        converter = Gst.ElementFactory.make("autovideoconvert", "conv")
        encoder = Gst.ElementFactory.make("x264enc", "encoder")
        muxcaps = Gst.ElementFactory.make("capsfilter", "capsmux")
        mux = Gst.ElementFactory.make("qtmux", "mux")
        
        filesave = Gst.ElementFactory.make("filesink", "sink")
        
        caps = Gst.Caps.from_string("video/x-raw,format=RGB,width=448,height=256,framerate=30/1")
        
        #Set properties of each element
        self.appsrc.set_property('caps', caps)

        muxcaps.set_property('caps', Gst.Caps.from_string("video/x-h264"))

        filesave.set_property("location", "video.qt")

        self.pipeline = Gst.Pipeline.new("test-pipeline")
        self.pipeline.add(self.appsrc)
        self.pipeline.add(converter)
        self.pipeline.add(encoder)
        self.pipeline.add(filesave)
        self.pipeline.add(muxcaps)
        self.pipeline.add(mux)
        
        #Linking pipeline elements
        self.appsrc.link(converter)
        converter.link(encoder)
        encoder.link(muxcaps)
        muxcaps.link(mux)
        mux.link(filesave)

    def update_buffer(self, image):
        
        #cv2.imwrite('test.jpg', image)

        if self.index < self.buffer_size:
            self.buffer[:, :, :, self.index] = image
            self.index += 1
        else:
            self.index = 0
            # return buffer to pipeline
            self.source_images()

    def source_images(self):
        
        self.pipeline.set_state(Gst.State.PLAYING)
        duration = 10**9 / (30 / 1) 
        pts = 0
        
        for image in range(self.buffer_size):
           
            image_bytes = self.buffer[:, :, :, image].tobytes()
            
            gst_buffer = Gst.Buffer.new_wrapped(image_bytes)
            
            pts += duration 
            gst_buffer.pts = pts
            gst_buffer.duration = duration

            self.appsrc.emit("push-buffer", gst_buffer)

        self.appsrc.emit("end-of-stream")

I've tried different encoders/muxers but the file is still unreadable. Interestingly, this does not happen when I use videotestsrc as the source rather than appsrc, which led me to believe that there is some sort of mismatch in the pipeline format, but I can't figure out the specifics.

score 0 · Accepted Answer · answered Feb 28 '23 at 22:08

Your code is OK, you set the caps in Gst.Caps format, the pipeline you build is OK, the caps, and the buffer you push, The only missing thing to make it work it's to set the AppSrc format property to Gst.Format.TIME.

The available options are:

  format              : The format of the segment events and seek
                        flags: readable, writable
                        Enum "GstFormat" Default: 2, "bytes"
                           (0): undefined        - GST_FORMAT_UNDEFINED
                           (1): default          - GST_FORMAT_DEFAULT
                           (2): bytes            - GST_FORMAT_BYTES
                           (3): time             - GST_FORMAT_TIME
                           (4): buffers          - GST_FORMAT_BUFFERS
                           (5): percent          - GST_FORMAT_PERCENT

This property specifies the type of buffer the Segment handles. In this case the position of buffer is given by time, start position of 0 and end position of the total duration this output file will have, as we are talking of sinking video to a file. See Segments for more details.

So, go add this line into your code to make it work!

self.appsrc.set_property("format", Gst.Format.TIME)

Unable to open the output video file of my pipeline when using appsrc

1 Answers1