4

I'm having trouble getting an image overlay to show up over the loopback video. I'm able to do a loopback with the script with rtpEndpoint.connect(rtpEndpoint, function....) but when I try to add in the ImageOverlayFilter I get some errors which I haven't been able to resolve.

Error: Invalid value for 'MediaType'

"error" : {"code" : 40001,"data" : {"type" : "MARSHALL_ERROR"},"message" : "Invalid value for 'MediaType'"}

   var overlayparams = {
       id: "TESTTEST",
       uri: "file://tmp/media/sample.png",
       offsetXPercent: 0,
       offsetYPercent: 0,
       widthPercent: 0,
       heightPercent: 0,
       keepAspectRatio: true,
       center: true,
       mediaPipeline: pipeline
    }

    pipeline.create("ImageOverlayFilter", overlayparams, function (error, filter) {
       if (error) return console.log(">>>>>"+ error);

       return callback(null, rtpEndpoint, filter);

    });

later in the file

    rtpEndpoint.connect(rtpEndpoint, imageOverlay, rtpEndpoint, function (error) {
        if (error) return onError(error);
        console.log("Added overlay to loopback"); 
    });

    /* 
    // This works for normal loopback
    rtpEndpoint.connect(rtpEndpoint, function (error) {
        if (error) return onError(error);
        console.log("loopback works"); 
    }); 
     */

I've also tried using the filter.addVideo method instead of the overlayparams object but it produces the same results.

Woodsy
  • 3,177
  • 2
  • 26
  • 50
  • 1
    Not to be a baby, but when and if you choose your answer please be sure to note that Deep Kakkar just reposted my exact same answer worded differently. I use kurento a lot and am not some hack throwing using other people answers or work. – ESportsVRGuy Feb 25 '20 at 01:00
  • I noticed that. I haven't had a chance to test your solution yet. – Woodsy Feb 25 '20 at 01:02
  • 1
    No worries! I'm just happy you noticed. Good luck! – ESportsVRGuy Feb 25 '20 at 03:03

1 Answers1

2

Updated! Assuming the imageoverlayfilter is anything like the faceoverlayfilter in Kurento the file referenced must be a http(s) url. If you are storing the files locally you will need to server them up using a webserver or host them remotely.

So, instead of... "file://tmp/media/sample.png"

Depending on where you have the root of your http(s) server You will have... "https://localhost/tmp/media/sample.png"

Also please note that there were issues with imageoverlay filter confirmed until kurento 6.11 . Latest is 6.13 so it could be solved... https://github.com/Kurento/bugtracker/issues/350

In addition to the described changes above you can try using the code below replacing FaceOverlayFilter to ImageOverlayFilter and setOverlayedImage to addImage as the usage and implementation of both filters is nearly the same.

So change...

pipeline.create("FaceOverlayFilter", (error, faceOverlayFilter) => {
            if (error) {
                return callback(error);
            }
            // This adds the Mario hat
            faceOverlayFilter.setOverlayedImage(
                url.format(asUrl) + "img/mario-wings.png",
                -0.35,
                -1.2,
                1.6,
                1.6,
                function(error) {
                    if (error) {
                        return callback(error);
                    }
                    return callback(null, webRtcEndpoint, faceOverlayFilter);
                }
            );
        });

To...

pipeline.create("ImageOverlayFilter ", (error, ImageOverlayFilter ) => {
            if (error) {
                return callback(error);
            }
            // This adds the image
            ImageOverlayFilter .addImage (
                "https://localhost/tmp/media/sample.png",
                0,
                0,
                0,
                0,
                true,
                true
                function(error) {
                    if (error) {
                        return callback(error);
                    }
                    return callback(null, webRtcEndpoint, ImageOverlayFilter );
                }
            );
        });
ESportsVRGuy
  • 101
  • 7
  • I'm still getting the same error with your fixes (using a hosted file vs local file) maybe it has to do with the version. If I figure it out I'll post the answer back here. – Woodsy Feb 25 '20 at 20:02
  • If you look at imageoverlay filter vs faceoverlay filter they inherit most of the same classes. Also their usage looks near identical. I would look at the magic mirror example here... https://github.com/Kurento/kurento-tutorial-node/blob/master/kurento-magic-mirror/server.js Then replace the faceOverlayFilter.setOverlayedImage calls with ImageOverlayFilter.addImage since the usage is practically identical as far as I can tell. The way you are binding it to the mediapipeline in your code might be outdated. Also, I am going to add a code snippet to my answer you can try. – ESportsVRGuy Feb 25 '20 at 20:46
  • Thanks for everything but its still not working... Now its saying `'operationParams' is required` Looks like this is similar to a known issue: https://github.com/Kurento/bugtracker/issues/350 – Woodsy Feb 26 '20 at 14:28
  • @Woodsy Haha! Yes, that's the bug and it explains the issue exactly. Did you already test it on Kurento 6.13 yet? If so, I can add a comment in the kurento github bugtracker indicating it;s still not fixed. Let me know. Thanks! – ESportsVRGuy Feb 27 '20 at 05:21
  • @Woodsy Also looks like they are working on Kurento 7.0 already and their integrating the new GStreamer should likely solve your issue. You can also supposedly download a preview to test if you are curious... https://groups.google.com/forum/#!topic/kurento/VFNHvJHSxUQ – ESportsVRGuy Feb 27 '20 at 05:32