How do I access the aspect ratio of a remote WebRTC video track through an Android custom renderer implementation using the SurfaceViewRenderer?

Question

The issue I am having is that my Xamarin Forms custom renderer for the Android SurfaceViewRenderer does not have the same aspect ratio as the remote video it displays. This causes the video to have an equal portion of both sides cut off. I have tried looking at the SurfaceChanged event of the SurfaceViewRenderer but the width and height are related to the SurfaceView itself and not the video track that it is the sink to.

In my OnAddStream function of my peer connection observer I can see the MediaStream object and the related remote VideoTrack. Here is the area of code I believe will be most relevant:

public void OnAddStream(MediaStream p0)
    {
        System.Diagnostics.Debug.WriteLine($"{nameof(OnAddStream)}");
        try
        {
            var videoTracks = p0.VideoTracks.OfType<VideoTrack>(); 
            videoTracks.FirstOrDefault().AddSink(_remoteView);

            var theRemoteSurface = _remoteView.Holder.SurfaceFrame;

            var metrics = Resources.System.DisplayMetrics;

            // this is wrong and should be the remote video aspect ratio
            decimal remoteRatio = decimal.Divide((theRemoteSurface.Right - theRemoteSurface.Left), (theRemoteSurface.Bottom - theRemoteSurface.Top));


            decimal myRatio = decimal.Divide(metrics.WidthPixels, metrics.HeightPixels);
            

            var audioTracks = p0.AudioTracks.OfType<AudioTrack>();
            audioTracks.FirstOrDefault().SetEnabled(true);
            audioTracks.FirstOrDefault().SetVolume(10);
        }
        catch (Exception ex)
        {

        }
    }

score 2 · Accepted Answer · answered Jan 26 '21 at 19:00

2

I found a solution to the problem. Instead of creating a custom renderer for the SurfaceViewRenderer within the WebRTC android package, I created one for Android.Widget.RelativeLayout. After making the RelativeLayout object I created a SurfaceViewRenderer object and used the AddView method to add it to the RelativeLayout. I then set the native control to the RelativeLayout and this allowed the aspect ratio to carry over.

answered Jan 26 '21 at 19:00

Matthew Jackson

69
6

Weird, I'll try it. – Jaswant Singh Aug 08 '22 at 19:34
Edit: This is the only method that worked! – Jaswant Singh Aug 08 '22 at 19:58
Glad it worked for you! – Matthew Jackson Aug 24 '22 at 13:44

How do I access the aspect ratio of a remote WebRTC video track through an Android custom renderer implementation using the SurfaceViewRenderer?

1 Answers1