1

I have an application in which I have implemented audio/video calling between web ( Reactjs ) and mobile ( React Native ). I am using react-native-webrtc for mobile side peer. When an audio call is connected and peer connections are created, the audio stream works perfectly. When I switch to video call during the audio call and the camera on my device turns on, the existing stream is not updated to accommodate both audio and video streams. I am looking for a solution where I can switch between audio and video calls on the fly and update the webRTC stream accordingly.

Reactjs code: Getting audio stream and setting it to localStream

getStream = (options) => {
navigator.getUserMedia =
  navigator.getUserMedia ||
  navigator.webkitGetUserMedia ||
  navigator.mozGetUserMedia;
navigator.getUserMedia(
  { audio: true, video: false },
  (stream) => {
    console.log("getusermedia event obj", stream);
    console.log(stream.getTracks());
    this.localGetTracks();
    if (this.localVideo.current !== null) {
      this.localVideo.current.srcObject = stream;
      this.localStream = stream;
    }
  },
  (error) => {
    console.log("User Media Error", error);
  }
)};

Reactjs code: New peer connection is created and stream is added with the localStream in it, which includes audio as true and video as false

newPeerConnection = () => {
window.RTCPeerConnection =
  window.mozRTCPeerConnection ||
  window.webkitRTCPeerConnection ||
  window.RTCPeerConnection;
let peerConn = new RTCPeerConnection({
  iceServers: turnServer,
});
peerConn.onicecandidate = (evt) => {
  if (evt.candidate) {
    this.props.connection.invoke(
      "addIceCandidate",
      parseInt(this.props.ticket.id),
      JSON.stringify({
        type: "candidate",
        sdpMLineIndex: evt.candidate.sdpMLineIndex,
        sdpMid: evt.candidate.sdpMid,
        candidate: evt.candidate.candidate,
      })
    );
  } else {
    console.log("End of candidates.");
    this.setState(
      {
        videoState: true,
        end_call: true,
        call_status: "Connected",
      },
      () => this.props._handleCallConnected(true)
    );

  }
  this.forceUpdate();
};
peerConn.addStream(this.localStream);
peerConn.addEventListener(
  "addstream",
  (stream) => {
    console.log("remoteStream on addstream", stream);
    this.remoteVideo.current.srcObject = stream.stream;

  },
  false
);

this.setState({
  peerConn: peerConn,
})};

React-native code: An event listener is subscribed for when a new stream is added, it listens to this eventListener and adds the upcoming stream to remoteStream

 peerConn.addEventListener(
  'addstream',
  (stream) => {
    console.log(`AddStreamListen`);

    console.log('remoteStream added', stream);

    InCallManager.setForceSpeakerphoneOn(false);
    this.setState({
      isSpeakerEnabled: false,
    });

    this.setState({
      remoteStream: stream,
      showAudioCallTimer: true,
    });
  },
  false,
);

Reactjs code: Adding a new stream with audio and video both as true

getVideoStream = () => {
navigator.getUserMedia =
  navigator.getUserMedia ||
  navigator.webkitGetUserMedia ||
  navigator.mozGetUserMedia;
navigator.getUserMedia(
  { audio: true, video: true, mirror: true },
  (stream) => {
    this.localStream = stream;
    this.localVideo.current.srcObject = this.localStream;
    this.state.peerConn.addStream(this.localStream);
  },
  (error) => {
    console.log("error", error);
  }
)};

But the eventListener on mobile code does not listen to the second stream I am adding with audio as true and video as true, the previous stream continues and video is not showing up

Avolox
  • 21
  • 4
  • Hmmm, I usually just create a blank mediastream and add the tracks after if they're coming in different order. You could await till both video/audio are ready to send both at once but I think you're mixed up somewhere. Tough to help. – BGPHiJACK Mar 04 '21 at 08:58
  • I don't want both audio video to be sent true the first time I add the stream, firstly I want the stream with only audio and then on an event of a video call button, I want to update the previous stream or add a new stream with audio and video both true – Avolox Mar 04 '21 at 10:03
  • Yup; define a blank media-stream and add to it as tracks populate. It can start out audio only, video only, take both or empty. – BGPHiJACK Mar 04 '21 at 10:05
  • The receiver peer does not listen to the addTrack or addStream event listener when I update or add a new stream or track from the sender peer – Avolox Mar 04 '21 at 10:16
  • You may need to write up a few small examples to get comfortable that's not right, but my answer stands, for that mobility of media you can start the mediastream blank, with audio/video and add to it later when emit of track is made and ready to be added. Perhaps try a ready made solution to simplify this. – BGPHiJACK Mar 04 '21 at 10:24
  • Yes I tried your solution, but when I add to it later, the receiver peer does not listen to the eventListener addstream or addtrack, then how would that peer add the upcoming stream to its remoteStream – Avolox Mar 04 '21 at 11:20
  • RTC is mostly asynchronous so it's possible you're doing this all out of step. To simplify if it helps, Connection is made with Peer A and B through it's typical process of passing/setting descriptions (signaling servers may exchange this information for you). During the exchange these listeners are established and when connected or ready Peer A sends stream, this will signal on Peer B's listeners that a track had came in to populate however. – BGPHiJACK Mar 04 '21 at 23:30

0 Answers0