1

Is there a way to play an external audio track in sync with a video, assuming they are the same length? The idea is to simulate a change in audio track like you would in a movie when changing languages. It seems like there isn't a native way. Btw, this is meant for browsers, mainly firefox and chrome.

I am unsure if there is something that can be done with the audio context here or will I have to play a muted video and an audio tag while somehow making sure they are in sync?

Thanks!

ErIK
  • 13
  • 3
  • hi, might be of interest https://stackoverflow.com/questions/7880878/multiple-audio-tracks-for-html5-video – jspcal Nov 30 '22 at 02:09
  • Sounds like they are saying it could be done with audiocontext, but they aren't explaining how. – ErIK Nov 30 '22 at 03:19
  • @ErIK [AudioTracks](https://developer.mozilla.org/en-US/docs/Web/API/HTMLMediaElement/audioTracks) are not supported except in Safari browser. **(1)** You can try using MediaSource Extensions (MSE) API to play custom or mixed a/v content. **(2)** Instead of generic "audios" and "videos" try to stick one actual format (like have one **Fragmented MP4** video with multiple **MP3** audios) then it's easier to learn what must be extracted from the format (video frame vs audio frame)... – VC.One Nov 30 '22 at 16:01
  • @ErIK Anyways if you want to try MSE then prepare your material (a/v file inputs) and then ask a new question when you can use it to play 1 video + 1 audio and just need advice about changing the audio part sent to the media decoder... – VC.One Nov 30 '22 at 16:05

1 Answers1

1

There is an old and meanwhile deprecated API which was meant for doing exactly that. It's called MediaController and there is still an implementation in Safari today.

Assigning both media elements to the same controller should be enough to sync them with each other.

<video controller="audio-and-video" src="video.file"></video>
<audio controller="audio-and-video" src="audio.file"></audio>

There is a more flexible thing called a TimingObject that can be used as well. It's not implemented in any browser but can be implemented in pure JavaScript. Let's imagine the HTML is the same as above.

<video id="video" src="video.file"></video>
<audio id="audio" src="audio.file"></audio>

It's then possible to use a bit of JavaScript to sync both media elements to the same TimingObject.

import { TimingObject } from 'timing-object';
import { setTimingsrc } from 'timingsrc';

const videoElement = document.getElementsById('video');
const audioElement = document.getElementsById('audio');

const timingObject = new TimingObject();

setTimingsrc(videoElement, timingObject);
setTimingsrc(audioElement, timingObject);

Instead of calling play() and pause() on one of the media elements you would then control the playback by updating the TimingObject.

// play
timingObject.update({ velocity: 1 });

// pause
timingObject.update({ velocity: 0 });
chrisguttandin
  • 7,025
  • 15
  • 21