Background
I am building a local area network, WebRTC baby monitor with a raspberry pi camera module and USB microphone. The stream is synthesized with GStreamer and im using Janus Gateway to facilitate the WebRTC connection between a web browser and the Pi. The webpage and Javascript is a stripped down version of the streaming demo provided by Meetecho.
This is my first time using many of these technologies and am a bit in over my head with troubleshooting at the moment. I'm having trouble figuring out why the web page works in Chrome and Safari, but does not work in Firefox.
It works fine on Chrome and Safari:
In Firefox, the WebRTC connection seems to be successfully established and maintained (based on comparisons of the network traffic and console output between Chrome and Firefox), but the page seems to get caught up somewhere along the way:
Comparing the consoles
When comparing the console outputs of Chrome and Firefox, they are identical until this point where both consoles report Uncaught (in promise) DOMException: but for possibly different reasons?
- Firefox says
Uncaught (in promise) DOMException: The fetching process for the media resource was aborted by the user agent at the user's request. - Chrome says
Uncaught (in promise) DOMException: The play() request was interrupted by a new load request.
Are these the same errors with different "Hints"? Or are they actually different errors due to some underlying difference between the browsers?
Immediately after this error, Firefox diverges from Chrome by reporting Remote track removed.
Im unsure if I am doing something silly in the JS to cause this or if there is some nuance about Firefox that I am missing.
Other details that might be helpful?
Below is part of the html (index.html) and javascript (janus_stream.js) for the page, the pastebin link contains the whole janus_stream.js.
// We make use of this 'server' variable to provide the address of the
// REST Janus API. By default, in this example we assume that Janus is
// co-located with the web server hosting the HTML pages but listening
// on a different port (8088, the default for HTTP in Janus), which is
// why we make use of the 'window.location.hostname' base address. Since
// Janus can also do HTTPS, and considering we don't really want to make
// use of HTTP for Janus if your demos are served on HTTPS, we also rely
// on the 'window.location.protocol' prefix to build the variable, in
// particular to also change the port used to contact Janus (8088 for
// HTTP and 8089 for HTTPS, if enabled).
// In case you place Janus behind an Apache frontend (as we did on the
// online demos at http://janus.conf.meetecho.com) you can just use a
// relative path for the variable, e.g.:
//
// var server = "/janus";
//
// which will take care of this on its own.
//
//
// If you want to use the WebSockets frontend to Janus, instead, you'll
// have to pass a different kind of address, e.g.:
//
// var server = "ws://" + window.location.hostname + ":8188";
//
// Of course this assumes that support for WebSockets has been built in
// when compiling the server. WebSockets support has not been tested
// as much as the REST API, so handle with care!
//
//
// If you have multiple options available, and want to let the library
// autodetect the best way to contact your server (or pool of servers),
// you can also pass an array of servers, e.g., to provide alternative
// means of access (e.g., try WebSockets first and, if that fails, fall
// back to plain HTTP) or just have failover servers:
//
// var server = [
// "ws://" + window.location.hostname + ":8188",
// "/janus"
// ];
//
// This will tell the library to try connecting to each of the servers
// in the presented order. The first working server will be used for
// the whole session.
//
var server = null;
if(window.location.protocol === 'http:')
server = "http://" + window.location.hostname + ":8088/janus";
else
server = "https://" + window.location.hostname + ":8089/janus";
var janus = null;
var streaming = null;
var opaqueId = "streamingtest-"+Janus.randomString(12);
var bitrateTimer = null;
var spinner = true;
var simulcastStarted = false, svcStarted = false;
var selectedStream = null;
$(document).ready(function() {
// Initialize the library (all console debuggers enabled)
Janus.init({debug: "all", callback: function() {
// Use a button to start the demo
//$('#start').one('click', function() {
//$(this).attr('disabled', true).unbind('click');
// Make sure the browser supports WebRTC
if(!Janus.isWebrtcSupported()) {
bootbox.alert("No WebRTC support... ");
return;
}
// Create session
janus = new Janus(
{
server: server,
success: function() {
// Attach to Streaming plugin
janus.attach(
{
plugin: "janus.plugin.streaming",
opaqueId: opaqueId,
success: function(pluginHandle) {
$('#details').remove();
streaming = pluginHandle;
Janus.log("Plugin attached! (" + streaming.getPlugin() + ", id=" + streaming.getId() + ")");
// Setup streaming session
$('#update-streams').click(updateStreamsList);
updateStreamsList();
$('#start').removeAttr('disabled').html("Stop")
.click(function() {
$(this).attr('disabled', true);
clearInterval(bitrateTimer);
janus.destroy();
$('#streamslist').attr('disabled', true);
$('#watch').attr('disabled', true).unbind('click');
$('#start').attr('disabled', true).html("Bye").unbind('click');
});
},
error: function(error) {
Janus.error(" -- Error attaching plugin... ", error);
bootbox.alert("Error attaching plugin... " + error);
},
iceState: function(state) {
Janus.log("ICE state changed to " + state);
},
webrtcState: function(on) {
Janus.log("Janus says our WebRTC PeerConnection is " + (on ? "up" : "down") + " now");
},
onmessage: function(msg, jsep) {
Janus.debug(" ::: Got a message :::", msg);
var result = msg["result"];
if(result) {
if(result["status"]) {
var status = result["status"];
if(status === 'starting')
$('#status').removeClass('hide').text("Starting, please wait...").show();
else if(status === 'started')
$('#status').removeClass('hide').text("Started").show();
else if(status === 'stopped')
stopStream();
} else if(msg["streaming"] === "event") {
// Is simulcast in place?
var substream = result["substream"];
var temporal = result["temporal"];
if((substream !== null && substream !== undefined) || (temporal !== null && temporal !== undefined)) {
if(!simulcastStarted) {
simulcastStarted = true;
addSimulcastButtons(temporal !== null && temporal !== undefined);
}
// We just received notice that there's been a switch, update the buttons
updateSimulcastButtons(substream, temporal);
}
// Is VP9/SVC in place?
var spatial = result["spatial_layer"];
temporal = result["temporal_layer"];
if((spatial !== null && spatial !== undefined) || (temporal !== null && temporal !== undefined)) {
if(!svcStarted) {
svcStarted = true;
addSvcButtons();
}
// We just received notice that there's been a switch, update the buttons
updateSvcButtons(spatial, temporal);
}
}
} else if(msg["error"]) {
bootbox.alert(msg["error"]);
stopStream();
return;
}
if(jsep) {
Janus.debug("Handling SDP as well...", jsep);
var stereo = (jsep.sdp.indexOf("stereo=1") !== -1);
// Offer from the plugin, let's answer
streaming.createAnswer(
{
jsep: jsep,
// We want recvonly audio/video and, if negotiated, datachannels
media: { audioSend: false, videoSend: false, data: true },
customizeSdp: function(jsep) {
if(stereo && jsep.sdp.indexOf("stereo=1") == -1) {
// Make sure that our offer contains stereo too
jsep.sdp = jsep.sdp.replace("useinbandfec=1", "useinbandfec=1;stereo=1");
}
},
success: function(jsep) {
Janus.debug("Got SDP!", jsep);
var body = { request: "start" };
streaming.send({ message: body, jsep: jsep });
$('#watch').html("Stop").removeAttr('disabled').click(stopStream);
},
error: function(error) {
Janus.error("WebRTC error:", error);
bootbox.alert("WebRTC error... " + error.message);
}
});
}
},
onremotestream: function(stream) {
Janus.debug(" ::: Got a remote stream :::", stream);
var addButtons = false;
if($('#remotevideo').length === 1) {
addButtons = true;
//$('#stream').append('<video class="rounded centered hide" id="remotevideo" width="100%" height="100%" playsinline/>');
$('#remotevideo').get(0).volume = 0;
// Show the stream and hide the spinner when we get a playing event
$("#remotevideo").bind("playing", function () {
$('#waitingvideo').remove();
if(this.videoWidth)
$('#remotevideo').removeClass('hide').show();
if(spinner)
spinner.stop();
spinner = null;
var videoTracks = stream.getVideoTracks();
if(!videoTracks || videoTracks.length === 0)
return;
var width = this.videoWidth;
var height = this.videoHeight;
$('#curres').removeClass('hide').text(width+'x'+height).show();
if(Janus.webRTCAdapter.browserDetails.browser === "firefox") {
// Firefox Stable has a bug: width and height are not immediately available after a playing
setTimeout(function() {
var width = $("#remotevideo").get(0).videoWidth;
var height = $("#remotevideo").get(0).videoHeight;
$('#curres').removeClass('hide').text(width+'x'+height).show();
}, 2000);
}
});
}
Janus.attachMediaStream($('#remotevideo').get(0), stream);
$("#remotevideo").get(0).play();
$("#remotevideo").get(0).volume = 1;
var videoTracks = stream.getVideoTracks();
if(!videoTracks || videoTracks.length === 0) {
// No remote video
$('#remotevideo').hide();
if($('#stream .no-video-container').length === 0) {
$('#stream').append(
'<div class="no-video-container">' +
'<i class="fa fa-video-camera fa-5 no-video-icon"></i>' +
'<span class="no-video-text">No remote video available</span>' +
'</div>');
}
} else {
$('#stream .no-video-container').remove();
$('#remotevideo').removeClass('hide').show();
}
if(!addButtons)
return;
if(videoTracks && videoTracks.length &&
(Janus.webRTCAdapter.browserDetails.browser === "chrome" ||
Janus.webRTCAdapter.browserDetails.browser === "firefox" ||
Janus.webRTCAdapter.browserDetails.browser === "safari")) {
$('#curbitrate').removeClass('hide').show();
bitrateTimer = setInterval(function() {
// Display updated bitrate, if supported
var bitrate = streaming.getBitrate();
$('#curbitrate').text(bitrate);
// Check if the resolution changed too
var width = $("#remotevideo").get(0).videoWidth;
var height = $("#remotevideo").get(0).videoHeight;
if(width > 0 && height > 0)
$('#curres').removeClass('hide').text(width+'x'+height).show();
}, 1000);
}
},
ondataopen: function(data) {
Janus.log("The DataChannel is available!");
$('#waitingvideo').remove();
$('#stream').append(
'<input class="form-control" type="text" id="datarecv" disabled></input>'
);
if(spinner)
spinner.stop();
spinner = null;
},
ondata: function(data) {
Janus.debug("We got data from the DataChannel!", data);
$('#datarecv').val(data);
},
oncleanup: function() {
Janus.log(" ::: Got a cleanup notification :::");
$('#waitingvideo').remove();
$('#remotevideo').remove();
$('#datarecv').remove();
$('.no-video-container').remove();
$('#bitrate').attr('disabled', true);
$('#bitrateset').html('Bandwidth<span class="caret"></span>');
$('#curbitrate').hide();
if(bitrateTimer)
clearInterval(bitrateTimer);
bitrateTimer = null;
$('#curres').hide();
$('#simulcast').remove();
$('#metadata').empty();
$('#info').addClass('hide').hide();
simulcastStarted = false;
}
});
},
error: function(error) {
Janus.error(error);
bootbox.alert(error, function() {
window.location.reload();
});
},
destroyed: function() {
window.location.reload();
}
});
//});
}});
});
function updateStreamsList() {
$('#update-streams').unbind('click').addClass('fa-spin');
var body = { request: "list" };
Janus.debug("Sending message:", body);
streaming.send({ message: body, success: function(result) {
setTimeout(function() {
$('#update-streams').removeClass('fa-spin').click(updateStreamsList);
}, 500);
if(!result) {
bootbox.alert("Got no response to our query for available streams");
return;
}
if(result["list"]) {
$('#streams').removeClass('hide').show();
$('#streamslist').empty();
$('#watch').attr('disabled', true).unbind('click');
var list = result["list"];
Janus.log("Got a list of available streams");
if(list && Array.isArray(list)) {
list.sort(function(a, b) {
if(!a || a.id < (b ? b.id : 0))
return -1;
if(!b || b.id < (a ? a.id : 0))
return 1;
return 0;
});
}
Janus.debug(list);
for(var mp in list) {
Janus.debug(" >> [" + list[mp]["id"] + "] " + list[mp]["description"] + " (" + list[mp]["type"] + ")");
$('#streamslist').append("<li><a href='#' id='" + list[mp]["id"] + "'>" + list[mp]["description"] + " (" + list[mp]["type"] + ")" + "</a></li>");
}
$('#streamslist a').unbind('click').click(function() {
selectedStream = $(this).attr("id");
$('#streamset').html($(this).html()).parent().removeClass('open');
return false;
});
$('#watch').removeAttr('disabled').unbind('click').click(startStream);
}
}});
}
function getStreamInfo() {
$('#metadata').empty();
$('#info').addClass('hide').hide();
if(!selectedStream)
return;
// Send a request for more info on the mountpoint we subscribed to
var body = { request: "info", id: parseInt(selectedStream) || selectedStream };
streaming.send({ message: body, success: function(result) {
if(result && result.info && result.info.metadata) {
$('#metadata').html(result.info.metadata);
$('#info').removeClass('hide').show();
}
}});
}
function startStream() {
selectedStream = "1"
Janus.log("Selected video id #" + selectedStream);
if(!selectedStream) {
bootbox.alert("Select a stream from the list");
return;
}
$('#streamset').attr('disabled', true);
$('#streamslist').attr('disabled', true);
$('#watch').attr('disabled', true).unbind('click');
var body = { request: "watch", id: parseInt(selectedStream) || selectedStream};
streaming.send({ message: body });
// No remote video yet
$('#stream').append('<video class="rounded centered" id="waitingvideo" width="100%" height="100%" />');
if(spinner == null) {
var target = document.getElementById('stream');
spinner = new Spinner({top:100}).spin(target);
} else {
spinner.spin();
}
// Get some more info for the mountpoint to display, if any
getStreamInfo();
}
function stopStream() {
$('#watch').attr('disabled', true).unbind('click');
var body = { request: "stop" };
streaming.send({ message: body });
streaming.hangup();
$('#streamset').removeAttr('disabled');
$('#streamslist').removeAttr('disabled');
$('#watch').html("Watch or Listen").removeAttr('disabled').unbind('click').click(startStream);
$('#status').empty().hide();
$('#bitrate').attr('disabled', true);
$('#bitrateset').html('Bandwidth<span class="caret"></span>');
$('#curbitrate').hide();
if(bitrateTimer)
clearInterval(bitrateTimer);
bitrateTimer = null;
$('#curres').empty().hide();
$('#simulcast').remove();
simulcastStarted = false;
}
.......<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0"/>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<title>BabyPi Cam</title>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/webrtc-adapter/7.4.0/adapter.min.js" ></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/jquery/1.9.1/jquery.min.js" ></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/twitter-bootstrap/3.4.1/js/bootstrap.min.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/bootbox.js/5.4.0/bootbox.min.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/spin.js/2.3.2/spin.min.js"></script>
<script type="text/javascript" src="https://cdnjs.cloudflare.com/ajax/libs/toastr.js/2.1.4/toastr.min.js"></script>
<script type="text/javascript" src="janus.js" ></script>
<script type="text/javascript" src="janus_stream.js"></script>
<script>
$(function() {
$(".navbar-static-top").load("navbar.html", function() {
$(".navbar-static-top li.dropdown").addClass("active");
$(".navbar-static-top a[href='streamingtest.html']").parent().addClass("active");
});
$(".footer").load("footer.html");
});
</script>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/bootswatch/3.4.0/cerulean/bootstrap.min.css" type="text/css"/>
<link rel="stylesheet" href="css/demo.css" type="text/css"/>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/font-awesome/4.7.0/css/font-awesome.min.css" type="text/css"/>
<link rel="stylesheet" href="https://cdnjs.cloudflare.com/ajax/libs/toastr.js/2.1.4/toastr.min.css"/>
</head>
<body>
<div class="container">
<div class="col-md-12">
<div class="panel panel-default">
<div class="panel-heading">
<h3 class="panel-title">BabyPi Live Stream
<span class="label label-info" id="status"></span>
<span class="label label-primary" id="curres"></span>
<span class="label label-info" id="curbitrate" styple="display:inline;"></span>
</h3>
</div>
<div class="panel-body" id="stream">
<video class="rounded centered" id="remotevideo" width="100%" height="100%" playsinline controls muted></video>
</div>
</div>
</div>
</div>
</body>
</html>I am also using the janus.js API provided Meetecho
The Questions
- What am I doing wrong that prevents this from working in Firefox?
- What can the diverging console outputs tell me about what I am doing wrong?
- Do you have any suggestions on where to look / what to try to get this working in Firefox?
Any pointers or ideas are greatly appreciated! Please let me know if I can provide other information.
Thank you!
Update: Theory / Possible answer?
In an attempt to address the Uncaught (in promise) DOMException: The fetching process for the media resource was aborted by the user agent at the user's request. error, I changed video.play() to video.load(). This addressed the error, but the same Remote track removed and "No remote video" behavior persists.
In the meantime I may have discovered the more fundamental issue: The video stream from the Pi is H264, and from what I can tell, Firefox does not support this format? Perhaps this is the reason that I am having issues with firefox?
Can any of you confirm or deny this as the true issue?
