1

I'm building a video calling app using WebRTC which allows one peer to call another by selecting someone in the lobby. When peer A sends a call request, the other peer B can accept. At this point, WebRTC signaling starts:

  • Both peers get their local media using MediaDevices.getUserMedia()
  • Both peers create an RTCPeerConnection and attach event listeners
  • Both peers calls RTCPeerConnection.addTrack() to add their local media
  • One peer A (the impolite user) creates an offer, calls RTCPeerConnection.setLocalDescription() to set that offer as the local description, and sends it to the WebSocket server, which forwards it to the other peer B.
  • The other peer B receives this offer and adds calls RTCPeerConnection.setRemoteDescription() to record it as the remote description
  • The other peer B then creates an answer and transmits it again to the first peer A.

(Steps based on https://developer.mozilla.org/en-US/docs/Web/API/WebRTC_API/Connectivity)

This flow is almost working well. In 1 out of 10 calls, I receive no video/audio from one of the peers (while both peers have working local video). In such a case, I have noticed that the answer SDP contains a=recvonly while this should be a=sendrecv under normal circumstances. I have further determined that by the time the other peer receives the offer and needs to reply with an answer, the localMedia of this side has sometimes not yet been added, because MediaDevices.getUserMedia can take a while to complete. I have also confirmed this order of operations by logging and observing that the offer sometimes arrives before local tracks were added.

I'm assuming that I shouldn't send an answer before the local media has been added?

I'm thinking of two ways to fix this, but I am not sure which option is best, if any:

  • Create the RTCPeerConnection only after MediaDevices.getUserMedia() completes. In the meantime, when receiving an offer, there is no peer connection yet, so we save offers in a buffer to process them later once the RTCPeerConnection is created.
  • When receiving an offer, and there are no localMedia tracks yet, hold off on creating the answer until the localMedia tracks have been added.

I am having difficulties to decide which solution (or another) matches best with the "Perfect Negotiation" pattern.

Thanks in advance!

Stephen B
  • 141
  • 2
  • 13

1 Answers1

2

Yes, it is good to add the stream before creating an offer if you do it 'statically', but the best way to do it is to do it in the onnegotiationneeded event because the addtrack event triggers an onnegotiationneeded event. So you should add the stream and then use the createoffer inside onnegotiationneeded. As far as the answer you can do it before with no issues, but remember that a well-established connection will let you add/remove tracks with no problems (even after the SDP has been set). You didn't post any code but remember that you also MUST exchange ice candidates. The last piece of advice, remember that all of the above IS asynchronous! so you should use promises, and await until the description is set, only THEN create an offer/answer. Hopefully, this will help

Skin_phil
  • 596
  • 5
  • 18
  • Thank you! Especially your "a well-established connection will let you add/remove tracks with no problems" statement gave me the idea that I shouldn't focus so much on the order of operations. By doing this, I noticed that renegotiation wasn't working correctly. At some point I inserted a "is user polite" check before creating an offer in onnegotiationneeded. If they are the polite side, I return and do not send an offer. I added this in order to fix a bug with an older browser that did not support "rollback". In hindsight, this was a silly solution, and of course breaks renegotiation. – Stephen B Jun 18 '21 at 12:59
  • Summarized: No, I didn't necessarily need to add local media tracks before sending the answer. Renegotiation will take care of that. – Stephen B Jun 18 '21 at 12:59
  • Glad to help! :) – Skin_phil Jun 18 '21 at 15:21
  • //remember that you also MUST exchange ice candidates.// it's Partially true. Because, if your sdp contains the Candidates (what it does when candidate discovery is complete), you don't need to exchange the ice candidate expletly. – Mateen Mar 22 '22 at 13:19