Modify data in a stream using GStreamer

Question

I've been looking for a way to modify a GStreamer stream in order to display the quality (by showing the raw video next to the result of the subtraction between the raw video and the enc/dec stream for example). I've been learning GStreamer for a week and so far I was able to do what I was asked but right now I'm stuck.

I looked into the compositor element that seems to mix streams but I'm pretty sure it cannot do what I need.

Then I checked appsrc and appsink in some code. I tried to build a pipeline: filesrc - appsink - appsrc - filesink. But for obvious reasons it did not work. I browsed github projects but most of appsrc/appsink uses were just to programmaticaly do a task like reading a file.

Lastly I found someone with the same problem like me. He "solved" it by creating 2 pipelines (filesrc - appsink & appsrc - filesink) but he still got allocation errors. I was not even able to run the code he shared.

Does anyone have any idea on how to get it done in a clean way?

score 1 · Accepted Answer · answered Feb 17 '22 at 20:37

I found a way to modify a stream. And it basically was to create my own plugin because by creating a element I can modify the stream between the source pad and the sink pad of my element. If someone is interrested there is the documentation explaining how to create a plugin and here's my chain function where I modify the data between the pads:

static GstFlowReturn
gst_my_filter_chain (GstPad * pad, GstObject * parent, GstBuffer * buf)
{
  GstMyFilter *filter;

  filter = GST_MYFILTER (parent);

  if (filter->silent == FALSE) {
    GstMapInfo info;
    if(gst_buffer_map(buf, &info, GST_MAP_READWRITE)) {
      guint8* offset = info.data;
      int i = 0;
      while(i < info.maxsize) {
        guint8* pos = offset + i;
        guint8 c = 100; // Value that will be added to make the color lighter
        int cc = (int) (*pos + c); // Adding the value to the color
        guint8 color;
        if(cc > 255) { // Making sure the color stays valid
          color = 255;
        }
        else {
          color = (guint8) cc;
        }
        memset(pos, color, 1); // Setting the value
        i++;
      }
    }
    gst_buffer_unmap(buf, &info);
  }
  return gst_pad_push (filter->srcpad, buf);
}

It's quite rudimentary and only lighten the video's colors but I can modify a stream so it's only a matter of time before I get the thing I want done. Just hope it'll help someone

score 0 · Answer 2 · answered Feb 15 '22 at 19:07

0

Compositor may be the simplest solution for what you're trying. You would have to use tee in order to fork into 2 sub-pipelines, one for direct video, and another one going thru encoding/decoding (here using jpeg), and compose both:

gst-launch-1.0  videotestsrc ! video/x-raw,width=640,height=480,framerate=30/1 ! tee name=t    t. ! queue ! comp.sink_0    t. ! queue ! jpegenc ! jpegdec ! comp.sink_1    compositor name=comp sink_0::xpos=0 sink_1::xpos=640 ! autovideosink

answered Feb 15 '22 at 19:07

SeB

1,159
6
17

I already can do what you did. The thing I wanted was to subtract the raw stream to the enc/dec to show the differences. That may be achieved using compositor by playing with opacity I don't know, but subtraction is not the only way to display quality of a video. And from what I was asked I definitely need a way to change directly the data from the stream by applying some code – Yuutsuna Feb 17 '22 at 20:14

Modify data in a stream using GStreamer

2 Answers2