2

I am developing a 3-part system for a remote vehicle to connect to a base station via WebRTC, which will pipe video from the remote vehicle to a remote station (also using an RTC Peer Connection). So, the architecture looks like this:


Vehicle -- PC --> Base Station -- PC --> Remote Station


Where PC is a peer connection. Video is sent from vehicle to Base Station using RTCPeerConnection's addTrack method. The video track must then be sent from Base Station (using addTrack method) to Remote Station via Remote Station's onTrack handler. The problem is, NodeJS (which Base Station is implemented in) does not natively support MediaStreams (Which are only available via a 3rd party library) and I am not able to find a way to trigger the onTrack handler in Remote Station. If Vehicle-Remote Station connect, the video comes through perfectly, but the Base Station is not triggering the appropriate event in Remote Station when piping. I added the package to support media streams, but this produces an error trying to locally store the stream and/or track (see tempStream/tempTrack variables),and does not seem to resolve the issue of the ontrack handler not being triggered in the Remote Station and displaying the video.

Here is the handler for the vehicle adding a track to pc, and trying to pipe that to rcspc, the RTCPeerConnection object for Remote Station. Any tips or insight is greatly appreciated.

    //Event listener for new track
     /**
     * When pc receives a track (video), pipe it over to Remote Station
     * using the addTrack method. (Currently NOT piping the video appropriately)
     * @param {} event 
     */
    pc.ontrack = (event) =>{
        console.log("--------------------RECEIVED TRACK--------------------");
        if (event.streams && event.streams[0]) {
           console.log('received track from vehicle');
           //add these streams to a temporary holding place
           tempStream.push(event.streams);
           tempTrack = event.track;

          //checking that rcspc peer connection has configured the remote description/finished
           //signaling process
           if(rcspc.remoteDescription){
                //changing to tempStream/tempTrack does not resolve the issue here.
                rcspc.addTrack(event.track, event.streams[0]);
           }

        } else {
            if (!remoteStream) {
                remoteStream = new MediaStream();
                //videoElem.srcObject = remoteStream;
                console.log('created new remote stream!');
            }
            remoteStream.addTrack(event.track);
        }
    };
girlcode
  • 61
  • 8

1 Answers1

1

To onTrack to fire on the other side of rcspc connection, you need offer/answer exchange after you've called addTrack. Otherwise the receiving side won't know what codec is used for the media you sending.

From mozilla docs:

Note: Adding a track to a connection triggers renegotiation by firing a negotiationneeded event. See Starting negotiation for details.

So I would try creating a new offer on rcspc after you add the track. Then wait for a new answer and set remote description. Then onTrack should fire on RS-side.

Or you can do offer-answer exchange in onnegotiationneeded callback, should make no difference.

Ivan
  • 624
  • 4
  • 7
  • I have done that (adding an offer/answer for rcspc from Base Station (offer) to RS (Answer), and it was able to trigger an offer/answer, but gave the error: [DOMException [InvalidAccessError]: Failed to set remote answer sdp: Failed to apply the description for 0: Failed to set SSL role for the transport.] after giving the answer from RS to Base Station. Any thoughts? This is happening directly after an answer is received. – girlcode Feb 02 '23 at 17:21
  • 1
    See this: https://stackoverflow.com/a/45140948/4755595 . If you tested with Firefox, you could try checking it with Chrome. Also you could try workaround from the answer: ensuring peers don't change roles, so the one sending offer is always the same. – Ivan Feb 02 '23 at 17:59