3

I’m trying to develop a webrtc video call system between mobile devices and computers. The mobile device sends the video stream of his to the computer.

In the mobile device the UI only shows the video it is recordings via its camera, in android this all works great but in iOS (I tested with 2 different iphones with the latest iOS version) the video doesn’t get started and it shown as black thought it gets transmitted to the computer without problems.

I tried everything I found online, adding controls to the video element, removing the controls, adding autoplay playsinline..., requesting the getUserMedia unload and reproducing the stream based on an user gesture, but nothing seems to work and I don't even get any errors on the console of the mobile phone. I tested on different browsers safari and chrome. And for more weirdness sometimes it starts when going to the list of tabs you have open on the browser and you go back to the page

This is the code of the video element

<video class='user-virtual-assessment-video' id='virtualAssessmentVideo' autoplay muted playsinline>
            
</video>
<button id='startVirtualAssessment' class='action-button'>
  <svg ...>
</button>

And the code executed when the user clicks on the button

e.stopPropagation();
e.preventDefault();
const videoElement = document.getElementById('virtualAssessmentVideo')
videoElement.srcObject = stream;
videoElement.play()
startVirtualAssessment()

The function startVirtualAssessment basically starts the rtc connection, and the user media of the iPhone gets requested on page load and stored on the stream variable

navigator.mediaDevices.getUserMedia({video: { facingMode: 'environment', aspectRatio: {exact: 16/9}, frameRate: { ideal: 10, max: 15 } }}).then((streamObject) => {
    stream = streamObject;
  })

Maybe there's something obvious that I'm missing or not understanding but I don't really see the problem with the code.

Thanks a lot :)

oscar rodriguez
  • 31
  • 7
  • Try changing `aspectRatio: {exact: 16/9}` to `aspectRatio: {ideal: 16/9}`. iOS is a bit picky about contstraints matching its cameras. Also, Chrome on iOS is a wrapper around Mobile Safari, so they're the same thing as far as gUM goes. – O. Jones Apr 05 '21 at 11:38
  • Hello Jones, thanks for your answer, I tried making the change you proposed but the problem persists, If it was a problem regarding the constraints shouldn't I have also a problem in the video transmitted over webRTC?, I can clearly see the video in the computer but not in the phone – oscar rodriguez Apr 07 '21 at 20:39
  • I'm seeing this as well and would love to know if anyone has a solution. Works in all browsers except mobile safari (iPad, iPhone). I can see about 1 frame of video before it goes black. I can confirm that the video stream data is being sent over the network because others can see my video stream. – Duane J Jun 04 '21 at 21:10

1 Answers1

1

We think we've identified the cause of this issue: on mobile safari (only) there is a limitation that getUserMedia cannot be requested multiple times for the same media type.

If you call getUserMedia once for the user to see him/herself prior to connecting to others, and then let a webRTC library (such as mediasoup, in our case) call getUserMedia again, the first request is muted (causing a black screen) and the second request will then take priority (letting others see the user).

A workaround seems to be to .clone() the media stream once retrieved the first time, rather than calling getUserMedia a second time.

See also this webkit bug report: https://bugs.webkit.org/show_bug.cgi?id=179363

Duane J
  • 1,615
  • 1
  • 15
  • 22